tests: Rework tests/installed → tests/kola
authorColin Walters <walters@verbum.org>
Wed, 11 Mar 2020 20:00:14 +0000 (20:00 +0000)
committerColin Walters <walters@verbum.org>
Thu, 19 Mar 2020 16:01:57 +0000 (16:01 +0000)
Previously we made an effort to use the [Fedora Standard Test Interface](https://docs.fedoraproject.org/en-US/ci/standard-test-interface/).
This effort was not very successful; the primary thing that
it really died on is Ansible just didn't support rebooting
very well.  I think that's since gotten better, but even
then, Ansible wasn't the best thing for a test framework
for us anyways.

In the meantime Fedora CoreOS happened emphasizing Ignition
and not "post-hoc reconciliation" models like Ansible over
ssh.

And, [coreos-assembler](https://github.com/coreos/coreos-assembler) happened too.

Furthermore, we really need to test OSTree's interaction
with Ignition as we've invented several special things there.

Then most recently, I've been working on having
cosa/kola support running externally defined tests:
https://github.com/coreos/coreos-assembler/pull/1215

There's a lot of things to clean up after this but at least this
works for me:

```
$ cd /srv/fcos
$ cosa kola run -- --parallel 4 --output-dir tmp/kola -E ~/src/github/ostreedev/ostree/ 'ext.ostree.*'
```

NOTE: This *does not* drop ostree binaries into the target.  See:
https://github.com/coreos/coreos-assembler/pull/1252#issuecomment-600623315

This drops our dependency on Python in the installed tests, and
also fixes a few bugs that came up.

I disabled the `itest-bare-user-root.sh` one because it's
entangled with the shell script infrastructure for the unit tests.

48 files changed:
tests/installed/README.md [deleted file]
tests/installed/destructive-ansible.yml [deleted file]
tests/installed/destructive-unit.yml [deleted file]
tests/installed/destructive/README.md [deleted file]
tests/installed/destructive/itest-bare-root.sh [deleted file]
tests/installed/destructive/itest-deploy-selinux.sh [deleted file]
tests/installed/destructive/itest-label-selinux.sh [deleted file]
tests/installed/destructive/staged-deploy.yml [deleted file]
tests/installed/destructive/var-mount.yml [deleted file]
tests/installed/execute_batch.yml [deleted file]
tests/installed/libinsttest.sh [deleted file]
tests/installed/libtest-core.sh [deleted symlink]
tests/installed/nondestructive.yml [deleted file]
tests/installed/nondestructive/itest-bare-unit.sh [deleted file]
tests/installed/nondestructive/itest-bare-user-root.sh [deleted file]
tests/installed/nondestructive/itest-bareuser-nouserxattrs.sh [deleted file]
tests/installed/nondestructive/itest-payload-link.sh [deleted file]
tests/installed/nondestructive/itest-pull-space.sh [deleted file]
tests/installed/nondestructive/itest-pull.sh [deleted file]
tests/installed/nondestructive/itest-remotes.sh [deleted file]
tests/installed/nondestructive/libtest-core.sh [deleted symlink]
tests/installed/playbook-run.sh [deleted file]
tests/installed/provision.sh [deleted file]
tests/installed/run.sh [deleted file]
tests/installed/tasks/disable-all-rpmmd-repos.yml [deleted file]
tests/installed/tasks/install-git.yml [deleted file]
tests/installed/tasks/overlay-git.yml [deleted file]
tests/installed/tasks/query-host.yml [deleted file]
tests/installed/tasks/reboot.yml [deleted file]
tests/kola/README.md [new file with mode: 0644]
tests/kola/destructive/data [new symlink]
tests/kola/destructive/itest-bare-root.sh [new file with mode: 0755]
tests/kola/destructive/itest-deploy-selinux.sh [new file with mode: 0755]
tests/kola/destructive/itest-label-selinux.sh [new file with mode: 0755]
tests/kola/destructive/staged-deploy.sh [new file with mode: 0755]
tests/kola/destructive/var-mount.sh [new file with mode: 0755]
tests/kola/libinsttest.sh [new file with mode: 0644]
tests/kola/libtest-core.sh [new file with mode: 0644]
tests/kola/nondestructive/data [new symlink]
tests/kola/nondestructive/itest-bare-unit.sh [new file with mode: 0644]
tests/kola/nondestructive/itest-bare-user-root.sh [new file with mode: 0755]
tests/kola/nondestructive/itest-bareuser-nouserxattrs.sh [new file with mode: 0755]
tests/kola/nondestructive/itest-payload-link.sh [new file with mode: 0755]
tests/kola/nondestructive/itest-pull-space.sh [new file with mode: 0755]
tests/kola/nondestructive/itest-pull.sh [new file with mode: 0755]
tests/kola/nondestructive/itest-remotes.sh [new file with mode: 0755]
tests/kola/nondestructive/libtest-core.sh [new symlink]
tests/libtest-core.sh [changed from file to symlink]

diff --git a/tests/installed/README.md b/tests/installed/README.md
deleted file mode 100644 (file)
index b3d14a7..0000000
+++ /dev/null
@@ -1,15 +0,0 @@
-This directory holds tests that use the
-[Fedora Standard Test Interface](https://fedoraproject.org/wiki/CI/Standard_Test_Interface).
-
-The high level structure is that we take a qcow2 file, inject
-built RPMs into it, and then use Ansible to run tests.
-
-See `.papr.yml` for canonical usage.
-
-For local development, you should cache the qcow2 somewhere
-stable (outside of this git repo).  Also note that `../ci/build-rpms.sh`
-does *not* pick up uncommitted changes!  Stated more strongly, you
-currently need to run `build-rpms.sh` after every change.
-
-To run just a specific test, use e.g.:
-`env TEST_SUBJECTS=/path/to/qcow2 ./playbook-run.sh -e tests=.*pull nondestructive.yml`
diff --git a/tests/installed/destructive-ansible.yml b/tests/installed/destructive-ansible.yml
deleted file mode 100644 (file)
index 50c8fd7..0000000
+++ /dev/null
@@ -1,16 +0,0 @@
-# Ansible-based tests.
----
-- hosts: localhost
-  tags:
-  - atomic
-  remote_user: root
-  vars:
-    use_git_build: True
-    tests: "."
-  tasks:
-    - import_tasks: tasks/disable-all-rpmmd-repos.yml
-    - import_tasks: tasks/install-git.yml
-      when: use_git_build
-    - import_tasks: tasks/query-host.yml
-    - import_tasks: destructive/staged-deploy.yml
-    - import_tasks: destructive/var-mount.yml
diff --git a/tests/installed/destructive-unit.yml b/tests/installed/destructive-unit.yml
deleted file mode 100644 (file)
index e70561d..0000000
+++ /dev/null
@@ -1,42 +0,0 @@
-# This entrypoint right now just runs shell-script based tests
-# from destructive/.  Note that we `rpm-ostree usroverlay` git
-# builds.  So it's not supported to reboot in these tests.
-# These tests will be run serially, and can e.g. change deployments.
----
-- hosts: localhost
-  tags:
-  - atomic
-  remote_user: root
-  vars:
-    use_git_build: True
-    tests: "."
-  tasks:
-    - import_tasks: tasks/query-host.yml
-    - set_fact:
-        rpmostree_initial_deployment: "{{ rpmostree_status[\"deployments\"][0] }}"
-    - import_tasks: tasks/overlay-git.yml
-      when: use_git_build
-    # Next copy all of the tests/ directory
-    - name: Copy test data
-      synchronize: src=../../ dest=/root/tests/ archive=yes
-
-    - find:
-        paths: /root/tests/installed/destructive
-        patterns: "itest-*.sh"
-      register: all_tests
-    - set_fact:
-        selected_tests: "{{ all_tests.files|map(attribute='path') | select('match', tests) | list }}"
-    - assert:
-        that:
-          - "{{ selected_tests|length }} != 0"
-    - file: path=/root/logs state=directory
-    - block:
-        - name: Run destructive tests
-          shell: "{{ item }} &> /root/logs/$(basename {{ item }}).log"
-          with_items:
-            - "{{ selected_tests }}"
-      always:
-        - synchronize:
-            src: /root/logs/
-            dest: artifacts/installed-destructive
-            mode: pull
diff --git a/tests/installed/destructive/README.md b/tests/installed/destructive/README.md
deleted file mode 100644 (file)
index cafe605..0000000
+++ /dev/null
@@ -1,5 +0,0 @@
-This suite of tests is run from PAPR. Everything in here is destructive; it's
-recommended to only run them in disposable virtual machines.  This is done
-in `tests/fedora-str/sysinstalled-tests.yml`, which currently uses a single VM
-and runs the tests serially.  It's likely in the future this will be changed
-to do one VM per test.
diff --git a/tests/installed/destructive/itest-bare-root.sh b/tests/installed/destructive/itest-bare-root.sh
deleted file mode 100755 (executable)
index 055d873..0000000
+++ /dev/null
@@ -1,44 +0,0 @@
-#!/bin/bash
-
-# Tests of the "raw ostree" functionality using the host's ostree repo as uid 0.
-
-set -xeuo pipefail
-
-dn=$(dirname $0)
-. ${dn}/../libinsttest.sh
-
-echo "1..2"
-date
-
-cd /ostree/repo/tmp
-rm co -rf
-rm co-testref -rf
-ostree refs --delete testref
-ostree checkout -H ${host_refspec} co
-victim_symlink=/usr/bin/gtar  # Seems likely to stick around
-# Copy the link to avoid corrupting it
-cp co/${victim_symlink}{,.tmp}
-mv co/${victim_symlink}{.tmp,}
-# Add another xattr to a symlink and a directory, since otherwise this is unusual
-setfattr -n security.biometric -v iris co/${victim_symlink}
-setfattr -n security.crunchy -v withketchup co/usr/bin
-csum=$(ostree commit -b testref --link-checkout-speedup --tree=dir=co)
-ostree fsck
-ostree ls -X testref ${victim_symlink} > ls.txt
-assert_file_has_content ls.txt 'security\.biometric'
-ostree ls -X ${host_refspec} ${victim_symlink} > ls.txt
-assert_not_file_has_content ls.txt 'security\.biometric'
-ostree ls -X testref usr/bin > ls.txt
-assert_file_has_content ls.txt 'security\.crunchy'
-
-ostree checkout -H testref co-testref
-getfattr -n security.biometric co-testref/${victim_symlink} > xattr.txt
-assert_file_has_content xattr.txt 'security\.biometric="iris"'
-getfattr -n security.crunchy co-testref/usr/bin > xattr.txt
-assert_file_has_content xattr.txt 'security\.crunchy="withketchup"'
-
-rm co -rf
-rm co-testref -rf
-
-echo "ok xattrs"
-date
diff --git a/tests/installed/destructive/itest-deploy-selinux.sh b/tests/installed/destructive/itest-deploy-selinux.sh
deleted file mode 100755 (executable)
index 92f6993..0000000
+++ /dev/null
@@ -1,58 +0,0 @@
-#!/bin/bash
-
-# Verify our /etc merge works with selinux
-
-set -xeuo pipefail
-
-dn=$(dirname $0)
-. ${dn}/../libinsttest.sh
-
-date
-# Create a new deployment
-ostree admin deploy --karg-proc-cmdline ${host_refspec}
-new_deployment_path=/ostree/deploy/${host_osname}/deploy/${host_commit}.1
-
-# Test /etc directory mtime
-if ! test ${new_deployment_path}/etc/NetworkManager -nt /etc/NetworkManager; then
-    ls -al ${new_deployment_path}/etc/NetworkManager /etc/NetworkManager
-    fatal "/etc directory mtime not newer"
-fi
-
-# A set of files that have a variety of security contexts
-for file in fstab passwd exports hostname sysctl.conf yum.repos.d \
-            NetworkManager/dispatcher.d/hook-network-manager; do
-    if ! test -e /etc/${file}; then
-        continue
-    fi
-
-    current=$(cd /etc && ls -Z ${file})
-    new=$(cd ${new_deployment_path}/etc && ls -Z ${file})
-    assert_streq "${current}" "${new}"
-done
-
-# Cleanup
-ostree admin undeploy 0
-
-cd /ostree/repo/tmp
-ostree checkout --fsync=0 -H ${host_commit} test-label
-rm test-label/usr/lib/ostree-boot/vmlinuz*
-rm test-label/usr/lib/ostree-boot/initramfs*
-cd test-label/usr/lib/modules/*
-rm initramfs.img
-echo new initramfs > initramfs.img
-cd -
-ostree commit --link-checkout-speedup --selinux-policy=test-label -b test-label --consume --tree=dir=test-label
-
-ostree admin deploy --karg-proc-cmdline test-label
-
-# This captures all of the boot entries; it'd be slightly annoying
-# to try to figure out the accurate one, so let's just ensure that at least
-# one entry is boot_t.
-# https://bugzilla.redhat.com/show_bug.cgi?id=1536991
-ls -Z /boot/ostree/*/ > bootlsz.txt
-assert_file_has_content_literal bootlsz.txt 'system_u:object_r:boot_t:s0 vmlinuz-'
-assert_file_has_content_literal bootlsz.txt 'system_u:object_r:boot_t:s0 initramfs-'
-
-ostree admin undeploy 0
-ostree refs --delete test-label
-date
diff --git a/tests/installed/destructive/itest-label-selinux.sh b/tests/installed/destructive/itest-label-selinux.sh
deleted file mode 100755 (executable)
index 2a49285..0000000
+++ /dev/null
@@ -1,89 +0,0 @@
-#!/bin/bash
-
-# Test commit --selinux-policy
-
-set -xeuo pipefail
-
-dn=$(dirname $0)
-. ${dn}/../libinsttest.sh
-
-date
-cd /ostree/repo/tmp
-rm co -rf
-ostree checkout -H ${host_refspec} co
-testbin=co/usr/bin/foo-a-generic-binary
-assert_not_has_file "${testbin}"
-# Make a test binary that we label as shell_exec_t on disk, but should be
-# reset by --selinux-policy back to bin_t
-echo 'test foo' > ${testbin}
-chcon --reference co/usr/bin/true ${testbin}
-oldcon=$(getfattr --only-values -m security.selinux ${testbin})
-chcon --reference co/usr/bin/bash ${testbin}
-newcon=$(getfattr --only-values -m security.selinux ${testbin})
-assert_not_streq "${oldcon}" "${newcon}"
-ostree commit -b testbranch --link-checkout-speedup \
-       --selinux-policy co --tree=dir=co
-ostree ls -X testbranch /usr/bin/foo-a-generic-binary > ls.txt
-assert_file_has_content ls.txt ${oldcon}
-ostree fsck
-
-ostree refs --delete testbranch
-rm co -rf
-echo "ok commit with sepolicy"
-
-# Now let's check that selinux policy labels can be applied on checkout
-
-rm rootfs -rf
-if ostree checkout -H \
-    --selinux-policy / ${host_refspec} co; then
-  assert_not_reached "checked out with -H and --selinux-policy"
-fi
-# recommit just two binaries into a new branch with selinux labels stripped
-mkdir -p rootfs/usr/bin
-oldcon=$(getfattr --only-values -m security.selinux /usr/bin/bash)
-cp /usr/bin/{true,bash} rootfs/usr/bin
-newcon=$(getfattr --only-values -m security.selinux rootfs/usr/bin/bash)
-assert_not_streq "${oldcon}" "${newcon}"
-echo "ok checkout with sepolicy setup"
-
-ostree commit -b testbranch rootfs
-ostree checkout testbranch --selinux-policy / co
-newcon=$(getfattr --only-values -m security.selinux co/usr/bin/bash)
-assert_streq "${oldcon}" "${newcon}"
-rm co -rf
-echo "ok checkout with sepolicy"
-ostree checkout testbranch --selinux-policy / --subpath /usr/bin co
-newcon=$(getfattr --only-values -m security.selinux co/bash)
-assert_streq "${oldcon}" "${newcon}"
-rm co -rf
-echo "ok checkout with sepolicy and subpath"
-
-# now commit tree with mismatched leading dirs
-mkdir -p rootfs/subdir
-mv rootfs/{usr,subdir}
-ostree commit -b testbranch rootfs
-ostree checkout testbranch --selinux-policy / co
-newcon=$(getfattr --only-values -m security.selinux co/subdir/usr/bin/bash)
-assert_not_streq "${oldcon}" "${newcon}"
-rm co -rf
-ostree checkout testbranch --selinux-policy / \
-  --subpath subdir --selinux-prefix / co
-newcon=$(getfattr --only-values -m security.selinux co/usr/bin/bash)
-assert_streq "${oldcon}" "${newcon}"
-rm co -rf
-echo "ok checkout with sepolicy and selinux-prefix"
-
-# Now check that combining --selinux-policy with --skip-list doesn't blow up
-echo > skip-list.txt << EOF
-/usr/bin/true
-EOF
-ostree checkout testbranch --selinux-policy / --skip-list skip-list.txt \
-  --subpath subdir --selinux-prefix / co
-! test -f co/usr/bin/true
-test -f co/usr/bin/bash
-newcon=$(getfattr --only-values -m security.selinux co/usr/bin/bash)
-assert_streq "${oldcon}" "${newcon}"
-rm co -rf
-ostree refs --delete testbranch
-echo "ok checkout selinux and skip-list"
-date
diff --git a/tests/installed/destructive/staged-deploy.yml b/tests/installed/destructive/staged-deploy.yml
deleted file mode 100644 (file)
index cfd9165..0000000
+++ /dev/null
@@ -1,117 +0,0 @@
-# Test the deploy --stage functionality; first, we stage a deployment
-# reboot, and validate that it worked.
-
-# for now, until the preset propagates down
-- name: Start up path unit
-  shell: |
-    set -xeuo pipefail
-    systemctl enable --now ostree-finalize-staged.path
-- name: Write staged-deploy commit
-  shell: |
-    set -xeuo pipefail
-    export OSTREE_SYSROOT_DEBUG="test-staged-path"
-    cd /ostree/repo/tmp
-    # https://github.com/ostreedev/ostree/issues/1569
-    ostree checkout -H ${commit} t
-    ostree commit --no-bindings --parent="${commit}" -b staged-deploy -I --consume t
-    newcommit=$(ostree rev-parse staged-deploy)
-    orig_mtime=$(stat -c '%.Y' /sysroot/ostree/deploy)
-    systemctl show -p SubState ostree-finalize-staged.path | grep -q waiting
-    systemctl show -p ActiveState ostree-finalize-staged.service | grep -q inactive
-    systemctl show -p TriggeredBy ostree-finalize-staged.service | grep -q path
-    ostree admin deploy --stage staged-deploy | tee out.txt
-    if ! grep -q 'test-staged-path: Not running' out.txt; then
-      cat out.txt
-      exit 1
-    fi
-    systemctl show -p SubState ostree-finalize-staged.path | grep running
-    systemctl show -p ActiveState ostree-finalize-staged.service | grep active
-    new_mtime=$(stat -c '%.Y' /sysroot/ostree/deploy)
-    test "${orig_mtime}" != "${new_mtime}"
-    test -f /run/ostree/staged-deployment
-    ostree refs | grep -E -e '^ostree/' | while read ref; do
-      if test "$(ostree rev-parse ${ref})" = "${newcommit}"; then
-        touch deployment-ref-found
-      fi
-    done
-    test -f deployment-ref-found
-    rm deployment-ref-found
-    if ostree admin pin 0 2>err.txt; then
-      echo "Pinned staged deployment"; exit 1
-    fi
-    grep -qFe 'Cannot pin staged deployment' err.txt
-  environment:
-    commit: "{{ rpmostree_status['deployments'][0]['checksum'] }}"
-- include_tasks: ../tasks/reboot.yml
-- name: Check that deploy-staged service worked
-  shell: |
-    set -xeuo pipefail
-    rpm-ostree status
-    # Assert that the previous boot had a journal entry for it
-    journalctl -b "-1" -u ostree-finalize-staged.service | grep -q -e 'Transaction complete'
-    # And there should not be a staged deployment
-    test '!' -f /run/ostree/staged-deployment
-
-- name: Upgrade with staging
-  shell: |
-    set -xeuo pipefail
-    test '!' -f /run/ostree/staged-deployment
-    ostree admin deploy --stage staged-deploy
-    test -f /run/ostree/staged-deployment
-    origcommit=$(ostree rev-parse staged-deploy)
-    cd /ostree/repo/tmp
-    ostree checkout -H "${origcommit}" t
-    ostree commit --no-bindings --parent="${origcommit}" -b staged-deploy -I --consume t
-    newcommit=$(ostree rev-parse staged-deploy)
-    env OSTREE_EX_STAGE_DEPLOYMENTS=1 ostree admin upgrade >out.txt
-    test -f /run/ostree/staged-deployment
-    # Debating bouncing back out to Ansible for this
-    firstdeploycommit=$(rpm-ostree status |grep 'Commit:' |head -1|sed -e 's,^ *Commit: *,,')
-    test "${firstdeploycommit}" = "${newcommit}"
-    # Cleanup
-    rpm-ostree cleanup -rp
-- import_tasks: ../tasks/query-host.yml
-
-# Ensure we can unstage
-- name: Write staged-deploy commit, then unstage
-  shell: |
-    set -xeuo pipefail
-    ostree admin deploy --stage staged-deploy
-    ostree admin status > status.txt
-    grep -qFe '(staged)' status.txt
-    test -f /run/ostree/staged-deployment
-    ostree admin undeploy 0
-    ostree admin status > status.txt
-    grep -vqFe '(staged)' status.txt
-    test '!' -f /run/ostree/staged-deployment
-
-- name: Staged should be overwritten by non-staged as first
-  shell: |
-    set -xeuo pipefail
-    ostree admin deploy --stage staged-deploy
-    test -f /run/ostree/staged-deployment
-    ostree --repo=/ostree/repo refs --create nonstaged-deploy "${commit}"
-    ostree admin deploy nonstaged-deploy
-    ostree admin status > status.txt
-    grep -vqFe '(staged)' status.txt
-    test '!' -f /run/ostree/staged-deployment
-    ostree admin undeploy 0
-  environment:
-    commit: "{{ rpmostree_status['deployments'][0]['checksum'] }}"
-
-- name: Staged is retained when pushing rollback
-  shell: |
-    set -xeuo pipefail
-    ostree admin deploy --stage staged-deploy
-    test -f /run/ostree/staged-deployment
-    ostree admin deploy --retain-pending --not-as-default nonstaged-deploy
-    test -f /run/ostree/staged-deployment
-    ostree admin status > status.txt
-    grep -qFe '(staged)' status.txt
-    ostree admin undeploy 0
-    ostree admin undeploy 1
-  environment:
-    commit: "{{ rpmostree_status['deployments'][0]['checksum'] }}"
-
-- name: Cleanup refs
-  shell: ostree refs --delete staged-deploy nonstaged-deploy 
diff --git a/tests/installed/destructive/var-mount.yml b/tests/installed/destructive/var-mount.yml
deleted file mode 100644 (file)
index 3fe041a..0000000
+++ /dev/null
@@ -1,13 +0,0 @@
-# https://github.com/ostreedev/ostree/issues/1667
-- name: Set up /var as a mountpoint
-  shell: |
-    set -xeuo pipefail
-    cp -a /var /sysroot/myvar
-    touch /sysroot/myvar/somenewfile
-    echo '/sysroot/myvar /var none bind 0 0' >> /etc/fstab
-- include_tasks: ../tasks/reboot.yml
-- name: Check that /var mountpoint worked
-  shell: |
-    set -xeuo pipefail
-    systemctl status var.mount
-    test -f /var/somenewfile
diff --git a/tests/installed/execute_batch.yml b/tests/installed/execute_batch.yml
deleted file mode 100644 (file)
index e4a11bb..0000000
+++ /dev/null
@@ -1,23 +0,0 @@
-#####################
-# execute_batch.yml
-#####################
-- name: Begin async command execution
-  shell: "{{ async_item }} &> {{ logdir }}/{{ async_item|basename }}.log"
-  # 10 minutes; the PAPR tester generally times out before that
-  async: 600
-  poll: 0
-  with_items: "{{ async_commands }}"
-  loop_control:
-    loop_var: "async_item"
-  register: async_results
-
-- name: Check async command status
-  async_status:
-    jid: "{{ async_result_item.ansible_job_id }}"
-  with_items: "{{ async_results.results }}"
-  loop_control:
-    loop_var: "async_result_item"
-  register: async_poll_results
-  until: async_poll_results.finished
-  retries: 500
-  retry_pause: 5
diff --git a/tests/installed/libinsttest.sh b/tests/installed/libinsttest.sh
deleted file mode 100644 (file)
index edf358f..0000000
+++ /dev/null
@@ -1,101 +0,0 @@
-# Common definitions for installed, privileged tests
-#
-# Copyright (C) 2017 Colin Walters <walters@verbum.org>
-#
-# SPDX-License-Identifier: LGPL-2.0+
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library; if not, write to the
-# Free Software Foundation, Inc., 59 Temple Place - Suite 330,
-# Boston, MA 02111-1307, USA.
-
-dn=$(dirname $0)
-. ${dn}/../libtest-core.sh
-
-# Copy of bits from tap-test
-test_tmpdir=
-function _tmpdir_cleanup () {
-    if test -z "${TEST_SKIP_CLEANUP:-}" &&
-       test -n "${test_tmpdir}" && test -f ${test_tmpdir}/.testtmp; then
-             rm "${test_tmpdir}" -rf
-    fi
-}
-prepare_tmpdir() {
-    local tmpdir=${1:-/tmp}
-    test_tmpdir=$(mktemp -p ${tmpdir} -d ostree-insttest.XXXXXXXX)
-    touch ${test_tmpdir}/.testtmp
-    cd ${test_tmpdir}
-}
-
-if test -x /usr/bin/python3; then
-    export PYTHON=/usr/bin/python3
-    export PYTHONHTTPSERVER=http.server
-elif test -x /usr/bin/python; then
-    export PYTHON=/usr/bin/python
-    export PYTHONHTTPSERVER=SimpleHTTPServer
-else
-    fatal "no python found"
-fi
-
-# This is copied from flatpak/flatpak/tests/test-webserver.sh
-run_tmp_webserver() {
-    dir=$1
-
-    test -n ${test_tmpdir}
-
-    cd ${dir}
-    env PYTHONUNBUFFERED=1 setsid $PYTHON -m $PYTHONHTTPSERVER 0 &>${test_tmpdir}/httpd-output &
-    cd -
-    child_pid=$!
-
-    for x in $(seq 60); do
-        echo "Waiting for web server ($x/60)..." >&2
-        # Snapshot the output
-        cp ${test_tmpdir}/httpd-output{,.tmp}
-        # If it's non-empty, see whether it matches our regexp
-        if test -s ${test_tmpdir}/httpd-output.tmp; then       # py3's http.server prints the http:// address also
-          sed -e 's,Serving HTTP on 0.0.0.0 port \([0-9]*\)\( (http://[^)]*)\)\? \.\.\.,\1,' < ${test_tmpdir}/httpd-output.tmp > ${test_tmpdir}/httpd-port
-            if ! cmp ${test_tmpdir}/httpd-output.tmp ${test_tmpdir}/httpd-port 1>/dev/null; then
-                # If so, we've successfully extracted the port
-                break
-            fi
-        fi
-        sleep 1
-    done
-
-    if [ ! -f ${test_tmpdir}/httpd-port ]; then
-      cat ${test_tmpdir}/httpd-output
-      fatal "can't start up httpd"
-    fi
-
-    port=$(cat ${test_tmpdir}/httpd-port)
-    echo "http://127.0.0.1:${port}" > ${test_tmpdir}/httpd-address
-    echo "$child_pid" > ${test_tmpdir}/httpd-pid
-}
-
-# Determine our origin refspec - we'll use this as a test base
-rpmostree=$(which rpm-ostree 2>/dev/null)
-if test -z "${rpmostree}"; then
-    skip "no rpm-ostree, at some point point this to raw ostree too"
-fi
-
-# We need to be root
-assert_streq $(id -u) 0
-
-rpmostree_query_json() {
-    query=$1
-    rpm-ostree status --json | $PYTHON -c 'import json,sys; v=json.load(sys.stdin); print(v'${query}')'
-}
-host_refspec=$(rpmostree_query_json '["deployments"][0]["origin"]')
-host_commit=$(rpmostree_query_json '["deployments"][0]["checksum"]')
-host_osname=$(rpmostree_query_json '["deployments"][0]["osname"]')
diff --git a/tests/installed/libtest-core.sh b/tests/installed/libtest-core.sh
deleted file mode 120000 (symlink)
index d26203e..0000000
+++ /dev/null
@@ -1 +0,0 @@
-../libtest-core.sh
\ No newline at end of file
diff --git a/tests/installed/nondestructive.yml b/tests/installed/nondestructive.yml
deleted file mode 100644 (file)
index dcd0d44..0000000
+++ /dev/null
@@ -1,41 +0,0 @@
-# Nondestructive sysinstalled tests, run in parallel.
----
-- hosts: localhost
-  tags:
-  - atomic
-  remote_user: root
-  vars:
-    use_git_build: True
-    tests: "."
-    # Arbitrary...we want some parallelism
-    batching_factor: 4
-  tasks:
-    - import_tasks: tasks/query-host.yml
-    - import_tasks: tasks/overlay-git.yml
-      when: use_git_build
-    # Next copy all of the tests/ directory
-    - name: Copy test data
-      synchronize: src=../../ dest=/root/tests/ archive=yes
-    - find:
-        paths: /root/tests/installed/nondestructive
-        patterns: "itest-*.sh"
-      register: all_tests
-    - set_fact:
-        selected_tests: "{{ all_tests.files|map(attribute='path') | select('match', tests) | list }}"
-    - assert:
-        that:
-          - "{{ selected_tests|length }} != 0"
-    - file: path=/root/logs state=directory
-    - block:
-        - name: Run nondestructive tests
-          vars:
-            logdir: /root/logs
-          async_commands: "{{ item }}"
-          include_tasks: execute_batch.yml
-          with_items:
-            - "{{ selected_tests | batch('{{ batching_factor }}') | list }}"
-      always:
-        - synchronize:
-            src: /root/logs
-            dest: artifacts/installed-nondestructive
-            mode: pull
diff --git a/tests/installed/nondestructive/itest-bare-unit.sh b/tests/installed/nondestructive/itest-bare-unit.sh
deleted file mode 100755 (executable)
index c973ee7..0000000
+++ /dev/null
@@ -1,41 +0,0 @@
-#!/bin/bash
-
-# Run test-basic.sh as root.
-# https://github.com/ostreedev/ostree/pull/1199
-
-set -xeuo pipefail
-
-dn=$(dirname $0)
-. ${dn}/../libinsttest.sh
-
-date
-# These tests sort of bypass the installed-tests spec;
-# fixing that would require installing g-d-t-r, though
-# more ideally we architect things with a "control" container
-# distinct from the host.
-export G_TEST_SRCDIR=$(realpath $dn/../../..)
-
-# Use /var/tmp to hopefully use XFS + O_TMPFILE etc.
-prepare_tmpdir /var/tmp
-trap _tmpdir_cleanup EXIT
-/usr/libexec/installed-tests/libostree/test-basic.sh
-/usr/libexec/installed-tests/libostree/test-basic-c
-date
-
-# Test error message when opening a non-world-readable object
-# https://github.com/ostreedev/ostree/issues/1562
-rm repo files -rf
-chmod a+rx .
-ostree --repo=repo init --mode=bare
-mkdir files
-touch files/unreadable
-chmod 0 files/unreadable
-ostree --repo=repo commit -b testbranch --tree=dir=files
-# We should be able to read as root due to CAP_DAC_OVERRIDE
-ostree --repo=repo cat testbranch /unreadable >/dev/null
-if setpriv --reuid bin --regid bin --clear-groups ostree --repo=repo cat testbranch /unreadable 2>err.txt; then
-    fatal "Listed unreadable object as non-root"
-fi
-assert_file_has_content err.txt "Opening content object.*openat: Permission denied"
-
-date
diff --git a/tests/installed/nondestructive/itest-bare-user-root.sh b/tests/installed/nondestructive/itest-bare-user-root.sh
deleted file mode 100755 (executable)
index 9e04f7b..0000000
+++ /dev/null
@@ -1,41 +0,0 @@
-#!/bin/bash
-
-# Tests of the "raw ostree" functionality using the host's ostree repo as uid 0.
-
-set -xeuo pipefail
-
-dn=$(dirname $0)
-. ${dn}/../libinsttest.sh
-
-echo "1..1"
-date
-
-prepare_tmpdir
-ostree --repo=repo init --mode=bare-user
-mkdir -p components/{dbus,systemd}/usr/{bin,lib}
-echo dbus binary > components/dbus/usr/bin/dbus-daemon
-chmod a+x components/dbus/usr/bin/dbus-daemon
-echo dbus lib > components/dbus/usr/lib/libdbus.so.1
-echo dbus helper > components/dbus/usr/lib/dbus-daemon-helper
-chmod a+x components/dbus/usr/lib/dbus-daemon-helper
-echo systemd binary > components/systemd/usr/bin/systemd
-chmod a+x components/systemd/usr/bin/systemd
-echo systemd lib > components/systemd/usr/lib/libsystemd.so.1
-
-# Make the gid on dbus 81 like fedora, also ensure no xattrs
-ostree --repo=repo commit --no-xattrs -b component-dbus --owner-uid 0 --owner-gid 81 --tree=dir=components/dbus
-ostree --repo=repo commit --no-xattrs -b component-systemd --owner-uid 0 --owner-gid 0 --tree=dir=components/systemd
-rm rootfs -rf
-for component in dbus systemd; do
-    ostree --repo=repo checkout -U -H component-${component} --union rootfs
-done
-echo 'some rootfs data' > rootfs/usr/lib/cache.txt
-# Commit using the host's selinux policy
-ostree --repo=repo commit --selinux-policy / -b rootfs --link-checkout-speedup --tree=dir=rootfs
-ostree --repo=repo ls rootfs /usr/bin/systemd >ls.txt
-assert_file_has_content ls.txt '^-007.. 0 0 .*/usr/bin/systemd'
-ostree --repo=repo ls -X rootfs /usr/lib/dbus-daemon-helper >ls.txt
-assert_file_has_content ls.txt '^-007.. 0 81 .*security\.selinux.*/usr/lib/dbus-daemon-helper'
-assert_not_file_has_content ls.txt 'user\.ostreemeta'
-echo "ok bare-user link-checkout-speedup with modified xattrs maintains uids"
-date
diff --git a/tests/installed/nondestructive/itest-bareuser-nouserxattrs.sh b/tests/installed/nondestructive/itest-bareuser-nouserxattrs.sh
deleted file mode 100755 (executable)
index 7063286..0000000
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/bash
-
-# Test that initializing a bare-user repo on tmpfs fails
-# Maybe at some point this will be fixed in the kernel
-# but I doubt it'll be soon
-# https://www.spinics.net/lists/linux-mm/msg109775.html
-
-set -xeuo pipefail
-
-dn=$(dirname $0)
-. ${dn}/../libinsttest.sh
-
-prepare_tmpdir
-trap _tmpdir_cleanup EXIT
-date
-
-mkdir mnt
-mount -t tmpfs tmpfs mnt
-if ostree --repo=mnt/repo init --mode=bare-user 2>err.txt; then
-    umount mnt
-    assert_not_reached "bare-user on tmpfs worked?"
-fi
-umount mnt
-assert_file_has_content err.txt "Operation not supported"
-date
diff --git a/tests/installed/nondestructive/itest-payload-link.sh b/tests/installed/nondestructive/itest-payload-link.sh
deleted file mode 100755 (executable)
index 6a6a01d..0000000
+++ /dev/null
@@ -1,141 +0,0 @@
-#!/bin/bash
-#
-# Copyright (C) 2018 Red Hat, Inc.
-#
-# SPDX-License-Identifier: LGPL-2.0+
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library; if not, write to the
-# Free Software Foundation, Inc., 59 Temple Place - Suite 330,
-# Boston, MA 02111-1307, USA.
-
-set -xeuo pipefail
-
-dn=$(dirname $0)
-. ${dn}/../libinsttest.sh
-
-echo "1..1"
-date
-
-# Use /var/tmp so we have O_TMPFILE etc.
-prepare_tmpdir /var/tmp
-trap _tmpdir_cleanup EXIT
-# We use this user down below, it needs access too
-setfacl -d -m u:bin:rwX .
-setfacl -m u:bin:rwX .
-ostree --repo=repo init --mode=archive
-echo -e '[archive]\nzlib-level=1\n' >> repo/config
-
-mkdir content
-cd content
-dd if=/dev/urandom of=bigobject bs=4k count=2560
-cp --reflink=auto bigobject bigobject2
-# Different metadata, same content
-chown bin:bin bigobject2
-cd ..
-ostree --repo=repo commit -b dupobjects --consume --selinux-policy=/ --tree=dir=content
-ostree --repo=repo summary -u
-
-run_tmp_webserver $(pwd)/repo
-
-origin=$(cat ${test_tmpdir}/httpd-address)
-
-cleanup() {
-    cd ${test_tmpdir}
-    for mnt in ${mnts:-}; do
-        umount ${mnt} || true
-    done
-    for blkdev in ${blkdevs:-}; do
-        losetup -d ${blkdev} || true
-    done
-}
-trap cleanup EXIT
-
-truncate -s 2G testblk1.img
-blkdev1=$(losetup --find --show $(pwd)/testblk1.img)
-blkdevs="${blkdev1}"
-# This filesystem must support reflinks
-mkfs.xfs -m reflink=1 ${blkdev1}
-mkdir mnt1
-mount ${blkdev1} mnt1
-mnts=mnt1
-
-truncate -s 2G testblk2.img
-blkdev2=$(losetup --find --show $(pwd)/testblk2.img)
-blkdevs="${blkdev1} ${blkdev2}"
-mkfs.xfs -m reflink=1 ${blkdev2}
-mkdir mnt2
-mount ${blkdev2} mnt2
-mnts="mnt1 mnt2"
-
-cd mnt1
-# See above for setfacl rationale
-setfacl -d -m u:bin:rwX .
-setfacl -m u:bin:rwX .
-ls -al .
-runuser -u bin mkdir foo
-runuser -u bin touch foo/bar
-ls -al foo
-
-# Test that reflink is really there (not just --reflink=auto)
-touch a
-cp --reflink a b
-mkdir repo
-ostree --repo=repo init
-ostree config --repo=repo set core.payload-link-threshold 0
-ostree --repo=repo remote add origin --set=gpg-verify=false ${origin}
-ostree --repo=repo pull --disable-static-deltas origin dupobjects
-find repo -type l -name '*.payload-link' >payload-links.txt
-assert_streq "$(wc -l < payload-links.txt)" "1"
-
-cat payload-links.txt | while read i; do
-    payload_checksum=$(basename $(dirname $i))$(basename $i .payload-link)
-    payload_checksum_calculated=$(sha256sum $(readlink -f $i) | cut -d ' ' -f 1)
-    assert_streq "${payload_checksum}" "${payload_checksum_calculated}"
-done
-echo "ok payload link"
-
-ostree --repo=repo checkout dupobjects content
-# And another object which differs just in metadata
-cp --reflink=auto content/bigobject{,3}
-chown operator:0 content/bigobject3
-cat >unpriv-child-repo.sh <<EOF
-#!/bin/bash
-# Check that commit to an user repo that has a parent still works
-set -xeuo pipefail
-ostree --repo=child-repo init --mode=bare-user
-ostree --repo=child-repo remote add origin --set=gpg-verify=false ${origin}
-ostree --repo=child-repo config set core.parent $(pwd)/repo
-ostree --repo=child-repo commit -b test content
-EOF
-chmod a+x unpriv-child-repo.sh
-runuser -u bin ./unpriv-child-repo.sh
-find child-repo -type l -name '*.payload-link' >payload-links.txt
-assert_streq "$(wc -l < payload-links.txt)" "0"
-rm content -rf
-
-echo "ok reflink unprivileged with parent repo"
-
-# We can't reflink across devices though
-cd ../mnt2
-ostree --repo=repo init --mode=archive
-ostree --repo=repo config set core.parent $(cd ../mnt1/repo && pwd)
-ostree --repo=../mnt1/repo checkout dupobjects content
-ostree --repo=repo commit -b dupobjects2 --consume --tree=dir=content
-ostree --repo=repo pull --disable-static-deltas origin dupobjects
-find repo -type l -name '*.payload-link' >payload-links.txt
-assert_streq "$(wc -l < payload-links.txt)" "0"
-
-echo "ok payload link across devices"
-
-date
diff --git a/tests/installed/nondestructive/itest-pull-space.sh b/tests/installed/nondestructive/itest-pull-space.sh
deleted file mode 100755 (executable)
index e365b86..0000000
+++ /dev/null
@@ -1,122 +0,0 @@
-#!/bin/bash
-# Test min-free-space-percent using loopback devices
-
-set -xeuo pipefail
-
-dn=$(dirname $0)
-. ${dn}/../libinsttest.sh
-date
-
-prepare_tmpdir
-trap _tmpdir_cleanup EXIT
-
-cd ${test_tmpdir}
-truncate -s 20MB testblk.img
-blkdev=$(losetup --find --show $(pwd)/testblk.img)
-mkfs.xfs ${blkdev}
-mkdir mnt
-mount ${blkdev} mnt
-
-# first test min-free-space-percent
-ostree --repo=mnt/repo init --mode=bare-user
-echo 'fsync=false' >> mnt/repo/config
-if ostree --repo=mnt/repo pull-local /ostree/repo ${host_commit} 2>err.txt; then
-    fatal "succeeded in doing a pull with no free space"
-fi
-assert_file_has_content err.txt "min-free-space-percent"
-echo "ok min-free-space-percent"
-
-# now test min-free-space-size
-rm -rf mnt/repo
-ostree --repo=mnt/repo init --mode=bare-user
-echo 'fsync=false' >> mnt/repo/config
-echo 'min-free-space-size=10MB' >> mnt/repo/config
-if ostree --repo=mnt/repo pull-local /ostree/repo ${host_commit} 2>err.txt; then
-    fatal "succeeded in doing a pull with no free space"
-fi
-assert_file_has_content err.txt "min-free-space-size"
-echo "ok min-free-space-size (error)"
-
-
-# min-free-space-size success
-ostree --repo=repo init --mode=bare-user
-echo 'fsync=false' >> repo/config
-echo 'min-free-space-size=1MB' >> repo/config
-ostree --repo=repo pull-local /ostree/repo ${host_commit}
-echo "ok min-free-space-size (success)"
-
-# metadata object write should succeed even if free space is lower than
-# min-free-space value
-rm -rf mnt/repo
-ostree --repo=mnt/repo init --mode=bare-user
-echo 'fsync=false' >> mnt/repo/config
-echo 'min-free-space-size=10MB' >> mnt/repo/config
-if ostree --repo=mnt/repo pull-local --commit-metadata-only /ostree/repo ${host_commit} 2>err.txt; then
-    fatal "could not write metadata objects even when min-free-space value should allow it"
-fi
-echo "ok metadata write even when free space is lower than min-free-space value"
-
-rm -rf mnt/repo
-
-# Test min-free-space-size on deltas
-
-#helper function copied from test-delta.sh
-get_assert_one_direntry_matching() {
-    local path=$1
-    local r=$2
-    local child=""
-    local bn
-    for p in ${path}/*; do
-  bn=$(basename $p)
-  if ! echo ${bn} | grep -q "$r"; then
-      continue
-  fi
-  if test -z "${child}"; then
-      child=${bn}
-  else
-      assert_not_reached "Expected only one child matching ${r} in ${path}";
-  fi
-    done
-    if test -z "${child}"; then
-  assert_not_reached "Failed to find child matching ${r}"
-    fi
-    echo ${child}
-}
-
-mkdir mnt/repo1 mnt/repo2
-ostree --repo=mnt/repo1 init --mode=bare-user
-ostree --repo=mnt/repo2 init --mode=bare-user
-mkdir files
-echo "hello" >> files/test.txt
-truncate -s 2MB files/test.txt
-
-host_commit=$(ostree --repo=mnt/repo1 commit -b test -s test --tree=dir=files)
-
-origrev=$(ostree --repo=mnt/repo1 rev-parse test)
-ostree --repo=mnt/repo1 static-delta generate --empty --to=${origrev}
-echo 'fsync=false' >> mnt/repo2/config
-echo 'min-free-space-size=12MB' >> mnt/repo2/config
-
-deltaprefix=$(get_assert_one_direntry_matching mnt/repo1/deltas '.')
-deltadir=$(get_assert_one_direntry_matching mnt/repo1/deltas/${deltaprefix} '.')
-
-# Try to pull delta and trigger error
-if ostree --repo=mnt/repo2 static-delta apply-offline mnt/repo1/deltas/${deltaprefix}/${deltadir} 2>err.txt; then
-    fatal "succeeded in doing a delta pull with no free space"
-fi
-assert_file_has_content err.txt "min-free-space-size"
-echo "OK min-free-space-size delta pull (error)"
-
-# Re-adjust min-free-space-size so that delta pull succeeds
-sed -i s/min-free-space-size=12MB/min-free-space-size=1MB/g mnt/repo2/config
-rm -rf mnt/repo2/deltas
-ostree --repo=mnt/repo2 static-delta apply-offline mnt/repo1/deltas/${deltaprefix}/${deltadir}
-
-echo "OK min-free-space-size delta pull (success)"
-rm -rf files
-
-umount mnt
-losetup -d ${blkdev}
-rm testblk.img
-
-date
diff --git a/tests/installed/nondestructive/itest-pull.sh b/tests/installed/nondestructive/itest-pull.sh
deleted file mode 100755 (executable)
index 07056ea..0000000
+++ /dev/null
@@ -1,83 +0,0 @@
-#!/bin/bash
-
-# Using the host ostree, test HTTP pulls
-
-set -xeuo pipefail
-
-# FIXME: https://github.com/ostreedev/ostree/pull/1548
-exit 0
-
-dn=$(dirname $0)
-. ${dn}/../libinsttest.sh
-date
-
-prepare_tmpdir /var/tmp
-trap _tmpdir_cleanup EXIT
-
-# Take the host's ostree, and make it archive
-mkdir repo
-ostree --repo=repo init --mode=archive
-echo -e '[archive]\nzlib-level=1\n' >> repo/config
-host_nonremoteref=$(echo ${host_refspec} | sed 's,[^:]*:,,')
-log_timestamps() {
-    date
-    "$@"
-    date
-}
-log_timestamps ostree --repo=repo pull-local /ostree/repo ${host_commit}
-ostree --repo=repo refs ${host_commit} --create=${host_nonremoteref}
-
-run_tmp_webserver $(pwd)/repo
-# Now test pulling via HTTP (no deltas) to a new bare-user repo
-ostree --repo=bare-repo init --mode=bare-user
-ostree --repo=bare-repo remote add origin --set=gpg-verify=false $(cat ${test_tmpdir}/httpd-address)
-log_timestamps ostree --repo=bare-repo pull --disable-static-deltas origin ${host_nonremoteref}
-
-echo "ok pull"
-
-# fsck marks commits partial
-# https://github.com/ostreedev/ostree/pull/1533
-for d in $(find bare-repo/objects/ -maxdepth 1 -type d); do
-    (find ${d} -name '*.file' || true) | head -20 | xargs rm -f
-done
-date
-if ostree --repo=bare-repo fsck |& tee fsck.txt; then
-    fatal "fsck unexpectedly succeeded"
-fi
-date
-assert_streq $(grep -cE -e 'Marking commit as partial' fsck.txt) "1"
-log_timestamps ostree --repo=bare-repo pull origin ${host_nonremoteref}
-# Don't need a full fsck here
-ostree --repo=bare-repo ls origin:${host_nonremoteref} >/dev/null
-
-rm bare-repo repo -rf
-
-# Try copying the host's repo across a mountpoint for direct
-# imports.
-cd ${test_tmpdir}
-mkdir tmpfs mnt
-mount --bind tmpfs mnt
-cd mnt
-ostree --repo=repo init --mode=bare
-log_timestamps ostree --repo=repo pull-local /ostree/repo ${host_commit}
-log_timestamps ostree --repo=repo fsck
-cd ..
-
-# Also, we shouldn't copy xattrs on metadata objects
-commit_path=objects/${host_commit:0:2}/${host_commit:2}.commit
-ostree --repo=testarchive init --mode=archive
-ostree --repo=testarchive pull-local --commit-metadata-only /ostree/repo ${host_commit}
-setfattr -n user.ostreetesting -v hello testarchive/${commit_path}
-ostree --repo=mnt/testarchive2 init --mode=archive
-ostree --repo=mnt/testarchive2 pull-local --commit-metadata-only testarchive ${host_commit}
-if getfattr -m user.ostreetesting mnt/testarchive2/${commit_path} 2>/dev/null; then
-    fatal "copied metadata xattr"
-fi
-echo "ok no metadata xattr copy"
-
-umount mnt
-
-# Cleanup
-kill -TERM $(cat ${test_tmpdir}/httpd-pid)
-echo "ok"
-date
diff --git a/tests/installed/nondestructive/itest-remotes.sh b/tests/installed/nondestructive/itest-remotes.sh
deleted file mode 100755 (executable)
index 836e35a..0000000
+++ /dev/null
@@ -1,18 +0,0 @@
-#!/bin/bash
-
-# Test that we didn't regress /etc/ostree/remotes.d handling
-
-set -xeuo pipefail
-
-dn=$(dirname $0)
-. ${dn}/../libinsttest.sh
-date
-
-prepare_tmpdir
-trap _tmpdir_cleanup EXIT
-
-ostree remote list > remotes.txt
-if ! test -s remotes.txt; then
-    assert_not_reached "no ostree remotes"
-fi
-date
diff --git a/tests/installed/nondestructive/libtest-core.sh b/tests/installed/nondestructive/libtest-core.sh
deleted file mode 120000 (symlink)
index d26203e..0000000
+++ /dev/null
@@ -1 +0,0 @@
-../libtest-core.sh
\ No newline at end of file
diff --git a/tests/installed/playbook-run.sh b/tests/installed/playbook-run.sh
deleted file mode 100755 (executable)
index af906c3..0000000
+++ /dev/null
@@ -1,41 +0,0 @@
-#!/usr/bin/bash
-# A thin wrapper for ansible-playbook which has a nice check for
-# TEST_SUBJECTS being set.
-set -xeuo pipefail
-
-dn=$(cd $(dirname $0) && pwd)
-if ! test -d build; then
-    mkdir -p build
-    (cd build && ${dn}/../../ci/build-rpm.sh)
-else
-    # XXX: we should invalidate based on `git describe` or something
-    echo "NOTE: Re-using prebuilt RPMs:"
-    find build -name '*.rpm'
-fi
-
-# https://fedoraproject.org/wiki/CI/Tests
-if test -z "${TEST_SUBJECTS:-}"; then
-    cat <<EOF
-
-error: TEST_SUBJECTS must be set; e.g.:
-
-  curl -Lo fedora-atomic-host.qcow2 'https://getfedora.org/atomic_qcow2_latest'
-  export TEST_SUBJECTS=\$(pwd)/fedora-atomic-host.qcow2
-
-If you're doing interactive development, we recommend caching the qcow2
-somewhere persistent.
-EOF
-    exit 1
-fi
-for subj in ${TEST_SUBJECTS}; do
-    ls -al ${subj} && file ${subj}
-done
-
-# This is required
-rpm -q standard-test-roles
-
-export ANSIBLE_INVENTORY=${ANSIBLE_INVENTORY:-$(test -e inventory && echo inventory || echo /usr/share/ansible/inventory)}
-ls -al /dev/kvm
-# Sadly having this on makes the reboot playbook break
-export ANSIBLE_SSH_ARGS='-o ControlMaster=no'
-exec ansible-playbook --tags=atomic "$@"
diff --git a/tests/installed/provision.sh b/tests/installed/provision.sh
deleted file mode 100755 (executable)
index b8ecdb4..0000000
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/usr/bin/bash
-set -xeuo pipefail
-
-dn=$(dirname $0)
-. ${dn}/../../ci/libpaprci/libbuild.sh
-
-pkg_upgrade
-pkg_install git rsync openssh-clients ansible standard-test-roles
diff --git a/tests/installed/run.sh b/tests/installed/run.sh
deleted file mode 100755 (executable)
index 2ee4a3d..0000000
+++ /dev/null
@@ -1,12 +0,0 @@
-#!/usr/bin/bash
-# Run all installed tests; see README.md in this directory for more
-# information.
-set -xeuo pipefail
-
-dn=$(cd $(dirname $0) && pwd)
-
-# TODO: parallelize this
-PLAYBOOKS=${PLAYBOOKS:-nondestructive.yml destructive-ansible.yml destructive-unit.yml}
-for playbook in $PLAYBOOKS; do
-    time ${dn}/playbook-run.sh -v ${dn}/${playbook}
-done
diff --git a/tests/installed/tasks/disable-all-rpmmd-repos.yml b/tests/installed/tasks/disable-all-rpmmd-repos.yml
deleted file mode 100644 (file)
index b4f2a07..0000000
+++ /dev/null
@@ -1,5 +0,0 @@
-- name: Disable all rpmmd repos
-  shell: |
-    for x in /etc/yum.repos.d/*.repo; do
-      sed -i -e 's,^enabled=,enabled=0,g' $x
-    done
diff --git a/tests/installed/tasks/install-git.yml b/tests/installed/tasks/install-git.yml
deleted file mode 100644 (file)
index 3374cce..0000000
+++ /dev/null
@@ -1,23 +0,0 @@
-# Use package layering to install git builds.
-- command: ostree --version
-  changed_when: False
-  register: ostree_orig_version
-- set_fact:
-    ostree_orig_version_yaml: "{{ ostree_orig_version.stdout | from_yaml }}"
-- name: Copy locally built RPMs
-  synchronize: src=build/x86_64/ dest=/root/x86_64/ archive=yes
-- name: Install RPMs
-  shell: rpm-ostree override replace /root/x86_64/*.rpm
-# Regenerate to make sure new ostree binaries also make it to the initrd
-- name: Regenerate initramfs
-  shell: rpm-ostree initramfs --enable
-- import_tasks: ../tasks/reboot.yml
-- import_tasks: ../tasks/query-host.yml
-- command: ostree --version
-  register: ostree_new_version
-- set_fact:
-    ostree_new_version_yaml: "{{ ostree_new_version.stdout | from_yaml }}"
-- name: "Fail if we didn't change the ostree version"
-  when: ostree_orig_version_yaml['libostree']['Git'] == ostree_new_version_yaml['libostree']['Git']
-  fail:
-    msg: "Failed to change ostree version"
diff --git a/tests/installed/tasks/overlay-git.yml b/tests/installed/tasks/overlay-git.yml
deleted file mode 100644 (file)
index 0018a1b..0000000
+++ /dev/null
@@ -1,23 +0,0 @@
-# Run "admin unlock" and add locally built RPMs, then
-# copy the whole tests/ directory into the VM.
-- command: ostree --version
-  changed_when: False
-  register: ostree_orig_version
-- set_fact:
-    ostree_orig_version_yaml: "{{ ostree_orig_version.stdout | from_yaml }}"
-- name: Copy locally built RPMs
-  synchronize: src=build/x86_64/ dest=/root/x86_64/ archive=yes
-- shell: ostree admin unlock || true
-# Install the RPMs we already have.  For the test suite we use rpm2cpio
-# since it depends on libsoup, but we're not using that yet for the sysinstalled tests
-- shell: >
-    /usr/bin/rpm -Fvh /root/x86_64/*.rpm && \
-    cd / && rpm2cpio /root/x86_64/ostree-tests-2*.rpm | cpio -div
-- command: ostree --version
-  register: ostree_new_version
-- set_fact:
-    ostree_new_version_yaml: "{{ ostree_new_version.stdout | from_yaml }}"
-- name: "Fail if we didn't change the ostree version"
-  when: ostree_orig_version_yaml['libostree']['Git'] == ostree_new_version_yaml['libostree']['Git']
-  fail:
-    msg: "Failed to change ostree version"
diff --git a/tests/installed/tasks/query-host.yml b/tests/installed/tasks/query-host.yml
deleted file mode 100644 (file)
index d572ae6..0000000
+++ /dev/null
@@ -1,6 +0,0 @@
-- name: Load status json
-  command: rpm-ostree status --json
-  changed_when: False
-  register: rpmostree_status_json
-- set_fact:
-    rpmostree_status: "{{ rpmostree_status_json.stdout | from_json }}"
diff --git a/tests/installed/tasks/reboot.yml b/tests/installed/tasks/reboot.yml
deleted file mode 100644 (file)
index a5d0cbe..0000000
+++ /dev/null
@@ -1,72 +0,0 @@
-# This file is copied from atomic-host-tests
-
-# vim: set ft=ansible:
-# There is no clean way to restart hosts in ansible. The general issue is that
-# the shutdown command may close sshd before ansible has time to "return" from
-# the task, even with async & poll. This is due to the fact that asynchronous
-# tasks still require a small synchronous bootstrapping script which takes 1 sec
-# to complete, during which it is vulnerable to erroring out if sshd dies.
-#       To mitigate this, we prefix a sleep command before the shutdown so
-# ansible has time to move on. For more info on this issue, see:
-# https://github.com/ansible/ansible/issues/10616
-#
-# The Ansible docs now recommend this combination of tasks to handle reboots
-# https://support.ansible.com/hc/en-us/articles/201958037-Reboot-a-server-and-wait-for-it-to-come-back
-
-# remember the real ansible_host for following local actions
-# (otherwise ansible will target the localhost)
-- set_fact:
-    real_ansible_host: "{{ ansible_host }}"
-    timeout: "{{ cli_reboot_timeout | default('120') }}"
-
-# Have to account for both because Fedora STR uses the old version of these
-# inventory values for some reason.
-- when: ansible_port is defined
-  set_fact:
-    real_ansible_port: "{{ ansible_port }}"
-
-- when: ansible_ssh_port is defined
-  set_fact:
-    real_ansible_port: "{{ ansible_ssh_port }}"
-
-- name: Get original bootid
-  command: cat /proc/sys/kernel/random/boot_id
-  register: orig_bootid
-
-# Stop sshd (thus preventing new connections) and kill our current user's
-# connection so that we can't race to get back in to the system while it's
-# shutting down
-- name: restart hosts
-  when: (not skip_shutdown is defined) or (not skip_shutdown)
-  shell: |
-    systemctl stop sshd
-    systemd-run --on-active=5 systemctl reboot
-  async: 1
-  poll: 0
-  ignore_errors: true
-
-# NB: The wait_for is executed locally and doesn't require privs, so avoid sudo
-- debug:
-    msg: "Waiting for reboot: {{ ansible_date_time.iso8601 }}"
-- wait_for_connection:
-    delay: 5
-    timeout: 120
-    search_regex: "OpenSSH"
-- debug:
-    msg: "SSH port is up {{ ansible_date_time.iso8601 }}"
-
-- name: Assert that the bootid changed
-  command: cat /proc/sys/kernel/random/boot_id
-  register: new_bootid
-  until: new_bootid.stdout != orig_bootid.stdout
-- assert:
-    that:
-      - new_bootid.stdout != orig_bootid.stdout
-
-# provide an empty iterator when a list is not provided
-# http://docs.ansible.com/ansible/playbooks_conditionals.html#loops-and-conditionals
-- name: check services have started
-  service:
-    name: "{{ item }}"
-    state: started
-  with_items: "{{ wait_for_services|default([]) }}"
diff --git a/tests/kola/README.md b/tests/kola/README.md
new file mode 100644 (file)
index 0000000..19cd552
--- /dev/null
@@ -0,0 +1 @@
+This uses https://github.com/coreos/coreos-assembler/pull/1252
diff --git a/tests/kola/destructive/data b/tests/kola/destructive/data
new file mode 120000 (symlink)
index 0000000..a96aa0e
--- /dev/null
@@ -0,0 +1 @@
+..
\ No newline at end of file
diff --git a/tests/kola/destructive/itest-bare-root.sh b/tests/kola/destructive/itest-bare-root.sh
new file mode 100755 (executable)
index 0000000..2834a82
--- /dev/null
@@ -0,0 +1,45 @@
+#!/bin/bash
+
+# Tests of the "raw ostree" functionality using the host's ostree repo as uid 0.
+
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+
+echo "1..2"
+date
+
+require_writable_sysroot
+
+cd /ostree/repo/tmp
+rm co -rf
+rm co-testref -rf
+ostree refs --delete testref
+ostree checkout -H ${host_refspec} co
+victim_symlink=/usr/bin/gtar  # Seems likely to stick around
+# Copy the link to avoid corrupting it
+cp co/${victim_symlink}{,.tmp}
+mv co/${victim_symlink}{.tmp,}
+# Add another xattr to a symlink and a directory, since otherwise this is unusual
+setfattr -n security.biometric -v iris co/${victim_symlink}
+setfattr -n security.crunchy -v withketchup co/usr/bin
+csum=$(ostree commit -b testref --link-checkout-speedup --tree=dir=co)
+ostree fsck
+ostree ls -X testref ${victim_symlink} > ls.txt
+assert_file_has_content ls.txt 'security\.biometric'
+ostree ls -X ${host_refspec} ${victim_symlink} > ls.txt
+assert_not_file_has_content ls.txt 'security\.biometric'
+ostree ls -X testref usr/bin > ls.txt
+assert_file_has_content ls.txt 'security\.crunchy'
+
+ostree checkout -H testref co-testref
+getfattr -n security.biometric co-testref/${victim_symlink} > xattr.txt
+assert_file_has_content xattr.txt 'security\.biometric="iris"'
+getfattr -n security.crunchy co-testref/usr/bin > xattr.txt
+assert_file_has_content xattr.txt 'security\.crunchy="withketchup"'
+
+rm co -rf
+rm co-testref -rf
+
+echo "ok xattrs"
+date
diff --git a/tests/kola/destructive/itest-deploy-selinux.sh b/tests/kola/destructive/itest-deploy-selinux.sh
new file mode 100755 (executable)
index 0000000..099b5c2
--- /dev/null
@@ -0,0 +1,59 @@
+#!/bin/bash
+
+# Verify our /etc merge works with selinux
+
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+
+require_writable_sysroot
+
+date
+# Create a new deployment
+ostree admin deploy --karg-proc-cmdline ${host_refspec}
+new_deployment_path=/ostree/deploy/${host_osname}/deploy/${host_commit}.1
+
+# Test /etc directory mtime
+if ! test ${new_deployment_path}/etc/NetworkManager -nt /etc/NetworkManager; then
+    ls -al ${new_deployment_path}/etc/NetworkManager /etc/NetworkManager
+    fatal "/etc directory mtime not newer"
+fi
+
+# A set of files that have a variety of security contexts
+for file in fstab passwd exports hostname sysctl.conf yum.repos.d \
+            NetworkManager/dispatcher.d/hook-network-manager; do
+    if ! test -e /etc/${file}; then
+        continue
+    fi
+
+    current=$(cd /etc && ls -Z ${file})
+    new=$(cd ${new_deployment_path}/etc && ls -Z ${file})
+    assert_streq "${current}" "${new}"
+done
+
+# Cleanup
+ostree admin undeploy 0
+
+cd /ostree/repo/tmp
+ostree checkout --fsync=0 -H ${host_commit} test-label
+rm -vf test-label/usr/lib/ostree-boot/vmlinuz*
+rm -vf test-label/usr/lib/ostree-boot/initramfs*
+cd test-label/usr/lib/modules/*
+rm initramfs.img
+echo new initramfs > initramfs.img
+cd -
+ostree commit --link-checkout-speedup --selinux-policy=test-label -b test-label --consume --tree=dir=test-label
+
+ostree admin deploy --karg-proc-cmdline test-label
+
+# This captures all of the boot entries; it'd be slightly annoying
+# to try to figure out the accurate one, so let's just ensure that at least
+# one entry is boot_t.
+# https://bugzilla.redhat.com/show_bug.cgi?id=1536991
+ls -Z /boot/ostree/*/ > bootlsz.txt
+assert_file_has_content_literal bootlsz.txt 'system_u:object_r:boot_t:s0 vmlinuz-'
+assert_file_has_content_literal bootlsz.txt 'system_u:object_r:boot_t:s0 initramfs-'
+
+ostree admin undeploy 0
+ostree refs --delete test-label
+date
diff --git a/tests/kola/destructive/itest-label-selinux.sh b/tests/kola/destructive/itest-label-selinux.sh
new file mode 100755 (executable)
index 0000000..87fb26f
--- /dev/null
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+# Test commit --selinux-policy
+
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+
+date
+cd /ostree/repo/tmp
+rm co -rf
+ostree checkout -H ${host_refspec} co
+testbin=co/usr/bin/foo-a-generic-binary
+assert_not_has_file "${testbin}"
+# Make a test binary that we label as shell_exec_t on disk, but should be
+# reset by --selinux-policy back to bin_t
+echo 'test foo' > ${testbin}
+chcon --reference co/usr/bin/true ${testbin}
+oldcon=$(getfattr --only-values -m security.selinux ${testbin})
+chcon --reference co/usr/bin/bash ${testbin}
+newcon=$(getfattr --only-values -m security.selinux ${testbin})
+assert_not_streq "${oldcon}" "${newcon}"
+ostree commit -b testbranch --link-checkout-speedup \
+       --selinux-policy co --tree=dir=co
+ostree ls -X testbranch /usr/bin/foo-a-generic-binary > ls.txt
+assert_file_has_content ls.txt ${oldcon}
+ostree fsck
+
+ostree refs --delete testbranch
+rm co -rf
+echo "ok commit with sepolicy"
+
+# Now let's check that selinux policy labels can be applied on checkout
+
+rm rootfs -rf
+if ostree checkout -H \
+    --selinux-policy / ${host_refspec} co; then
+  assert_not_reached "checked out with -H and --selinux-policy"
+fi
+# recommit just two binaries into a new branch with selinux labels stripped
+mkdir -p rootfs/usr/bin
+oldcon=$(getfattr --only-values -m security.selinux /usr/bin/bash)
+cp /usr/bin/{true,bash} rootfs/usr/bin
+newcon=$(getfattr --only-values -m security.selinux rootfs/usr/bin/bash)
+assert_not_streq "${oldcon}" "${newcon}"
+echo "ok checkout with sepolicy setup"
+
+ostree commit -b testbranch rootfs
+ostree checkout testbranch --selinux-policy / co
+newcon=$(getfattr --only-values -m security.selinux co/usr/bin/bash)
+assert_streq "${oldcon}" "${newcon}"
+rm co -rf
+echo "ok checkout with sepolicy"
+ostree checkout testbranch --selinux-policy / --subpath /usr/bin co
+newcon=$(getfattr --only-values -m security.selinux co/bash)
+assert_streq "${oldcon}" "${newcon}"
+rm co -rf
+echo "ok checkout with sepolicy and subpath"
+
+# now commit tree with mismatched leading dirs
+mkdir -p rootfs/subdir
+mv rootfs/{usr,subdir}
+ostree commit -b testbranch rootfs
+ostree checkout testbranch --selinux-policy / co
+newcon=$(getfattr --only-values -m security.selinux co/subdir/usr/bin/bash)
+assert_not_streq "${oldcon}" "${newcon}"
+rm co -rf
+ostree checkout testbranch --selinux-policy / \
+  --subpath subdir --selinux-prefix / co
+newcon=$(getfattr --only-values -m security.selinux co/usr/bin/bash)
+assert_streq "${oldcon}" "${newcon}"
+rm co -rf
+echo "ok checkout with sepolicy and selinux-prefix"
+
+# Now check that combining --selinux-policy with --skip-list doesn't blow up
+echo > skip-list.txt << EOF
+/usr/bin/true
+EOF
+ostree checkout testbranch --selinux-policy / --skip-list skip-list.txt \
+  --subpath subdir --selinux-prefix / co
+! test -f co/usr/bin/true
+test -f co/usr/bin/bash
+newcon=$(getfattr --only-values -m security.selinux co/usr/bin/bash)
+assert_streq "${oldcon}" "${newcon}"
+rm co -rf
+ostree refs --delete testbranch
+echo "ok checkout selinux and skip-list"
+date
diff --git a/tests/kola/destructive/staged-deploy.sh b/tests/kola/destructive/staged-deploy.sh
new file mode 100755 (executable)
index 0000000..47d7c9a
--- /dev/null
@@ -0,0 +1,118 @@
+#!/bin/bash
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+
+require_writable_sysroot
+prepare_tmpdir
+
+n=$(nth_boot)
+case "${n}" in
+  1)
+    commit=${host_commit}
+  # Test the deploy --stage functionality; first, we stage a deployment
+  # reboot, and validate that it worked.
+  # for now, until the preset propagates down
+  # Start up path unit
+    systemctl enable --now ostree-finalize-staged.path
+  # Write staged-deploy commit
+    export OSTREE_SYSROOT_DEBUG="test-staged-path"
+    cd /ostree/repo/tmp
+    # https://github.com/ostreedev/ostree/issues/1569
+    ostree checkout -H ${commit} t
+    ostree commit --no-bindings --parent="${commit}" -b staged-deploy -I --consume t
+    newcommit=$(ostree rev-parse staged-deploy)
+    orig_mtime=$(stat -c '%.Y' /sysroot/ostree/deploy)
+    systemctl show -p SubState ostree-finalize-staged.path | grep -q waiting
+    systemctl show -p ActiveState ostree-finalize-staged.service | grep -q inactive
+    systemctl show -p TriggeredBy ostree-finalize-staged.service | grep -q path
+    ostree admin deploy --stage staged-deploy | tee out.txt
+    assert_file_has_content out.txt 'test-staged-path: Not running'
+    systemctl show -p SubState ostree-finalize-staged.path | grep running
+    systemctl show -p ActiveState ostree-finalize-staged.service | grep active
+    new_mtime=$(stat -c '%.Y' /sysroot/ostree/deploy)
+    test "${orig_mtime}" != "${new_mtime}"
+    test -f /run/ostree/staged-deployment
+    ostree refs | grep -E -e '^ostree/' | while read ref; do
+      if test "$(ostree rev-parse ${ref})" = "${newcommit}"; then
+        touch deployment-ref-found
+      fi
+    done
+    test -f deployment-ref-found
+    rm deployment-ref-found
+    if ostree admin pin 0 2>err.txt; then
+      fatal "Pinned staged deployment"
+    fi
+    assert_file_has_content err.txt 'Cannot pin staged deployment'
+    kola_reboot
+    ;;
+  2) 
+    # Check that deploy-staged service worked
+    rpm-ostree status
+    # Assert that the previous boot had a journal entry for it
+    journalctl -b "-1" -u ostree-finalize-staged.service > svc.txt
+    assert_file_has_content svc.txt 'Bootloader updated; bootconfig swap: yes; deployment count change: 1'
+    rm -f svc.txt
+    # And there should not be a staged deployment
+    test '!' -f /run/ostree/staged-deployment
+
+    # Upgrade with staging
+    test '!' -f /run/ostree/staged-deployment
+    ostree admin deploy --stage staged-deploy
+    test -f /run/ostree/staged-deployment
+    origcommit=$(ostree rev-parse staged-deploy)
+    cd /ostree/repo/tmp
+    ostree checkout -H "${origcommit}" t
+    ostree commit --no-bindings --parent="${origcommit}" -b staged-deploy -I --consume t
+    newcommit=$(ostree rev-parse staged-deploy)
+    env OSTREE_EX_STAGE_DEPLOYMENTS=1 ostree admin upgrade >out.txt
+    test -f /run/ostree/staged-deployment
+    # Debating bouncing back out to Ansible for this
+    firstdeploycommit=$(rpm-ostree status |grep 'Commit:' |head -1|sed -e 's,^ *Commit: *,,')
+    assert_streq "${firstdeploycommit}" "${newcommit}"
+    # Cleanup
+    rpm-ostree cleanup -rp
+    echo "ok upgrade with staging"
+
+    # Ensure we can unstage
+    # Write staged-deploy commit, then unstage
+    ostree admin deploy --stage staged-deploy
+    ostree admin status > status.txt
+    assert_file_has_content_literal status.txt '(staged)'
+    test -f /run/ostree/staged-deployment
+    ostree admin undeploy 0
+    ostree admin status > status.txt
+    grep -vqFe '(staged)' status.txt
+    test '!' -f /run/ostree/staged-deployment
+    echo "ok unstage"
+
+   # Staged should be overwritten by non-staged as first
+    commit=$(rpmostree_query_json '.deployments[0].checksum')
+    ostree admin deploy --stage staged-deploy
+    test -f /run/ostree/staged-deployment
+    ostree --repo=/ostree/repo refs --create nonstaged-deploy "${commit}"
+    ostree admin deploy nonstaged-deploy
+    ostree admin status > status.txt
+    grep -vqFe '(staged)' status.txt
+    test '!' -f /run/ostree/staged-deployment
+    ostree admin undeploy 0
+    echo "ok staged overwritten by non-staged"
+
+  # Staged is retained when pushing rollback
+    commit=$(rpmostree_query_json '.deployments[0].checksum')
+    ostree admin deploy --stage staged-deploy
+    test -f /run/ostree/staged-deployment
+    ostree admin deploy --retain-pending --not-as-default nonstaged-deploy
+    test -f /run/ostree/staged-deployment
+    ostree admin status > status.txt
+    assert_file_has_content_literal status.txt '(staged)'
+    ostree admin undeploy 0
+    ostree admin undeploy 1
+    echo "ok staged retained"
+
+    # Cleanup refs
+    ostree refs --delete staged-deploy nonstaged-deploy
+    echo "ok cleanup refs"
+    ;;
+  *) fatal "Unexpected boot count" ;;
+esac
diff --git a/tests/kola/destructive/var-mount.sh b/tests/kola/destructive/var-mount.sh
new file mode 100755 (executable)
index 0000000..b3fe3d9
--- /dev/null
@@ -0,0 +1,23 @@
+#!/bin/bash
+# https://github.com/ostreedev/ostree/issues/1667
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+
+n=$(nth_boot)
+case "${n}" in
+  1) 
+    require_writable_sysroot
+    # Hack this off for now
+    chattr -i /sysroot
+    cp -a /var /sysroot/myvar
+    touch /sysroot/myvar/somenewfile
+    echo '/sysroot/myvar /var none bind 0 0' >> /etc/fstab
+    kola_reboot
+    ;;
+  2)
+    systemctl status var.mount
+    test -f /var/somenewfile
+    ;;
+  *) fatal "Unexpected boot count $n"
+esac
diff --git a/tests/kola/libinsttest.sh b/tests/kola/libinsttest.sh
new file mode 100644 (file)
index 0000000..2552cf7
--- /dev/null
@@ -0,0 +1,91 @@
+# Common definitions for installed, privileged tests
+#
+# Copyright (C) 2017 Colin Walters <walters@verbum.org>
+#
+# SPDX-License-Identifier: LGPL-2.0+
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the
+# Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+# Boston, MA 02111-1307, USA.
+
+. ${KOLA_EXT_DATA}/libtest-core.sh
+
+# Copy of bits from tap-test
+test_tmpdir=
+function _tmpdir_cleanup () {
+    if test -z "${TEST_SKIP_CLEANUP:-}" &&
+       test -n "${test_tmpdir}" && test -f ${test_tmpdir}/.testtmp; then
+             rm "${test_tmpdir}" -rf
+    fi
+}
+prepare_tmpdir() {
+    local tmpdir=${1:-/var/tmp}
+    test_tmpdir=$(mktemp -p ${tmpdir} -d ostree-insttest.XXXXXXXX)
+    touch ${test_tmpdir}/.testtmp
+    cd ${test_tmpdir}
+}
+
+# This is copied from flatpak/flatpak/tests/test-webserver.sh
+run_tmp_webserver() {
+    dir=$1
+
+    port=8000
+    podman create --name ostree-httpd --privileged  -ti --net=host -v "${dir}":/srv --workdir /srv \
+        registry.svc.ci.openshift.org/coreos/fedora:31 python3 -m http.server "${port}"
+    podman generate systemd ostree-httpd > /etc/systemd/system/ostree-httpd.service
+    systemctl daemon-reload
+    systemctl start ostree-httpd.service
+
+    address="http://127.0.0.1:${port}"
+    while ! curl --head "${address}" &>/dev/null; do sleep 1; done
+    echo "${address}" > ${test_tmpdir}/httpd-address
+}
+
+# Yeah this is a hack.  Doing this better requires more first class support
+# for creating derived commits in ostree potentially.  Or barring that,
+# just recommend to people to use `unshare -m` or equivalent and
+# mount -o remount,rw /sysroot in their code.
+require_writable_sysroot() {
+    if ! test -w /sysroot; then
+        mount -o remount,rw /sysroot
+    fi
+}
+
+nth_boot() {
+    journalctl --list-boots | wc -l
+}
+
+kola_reboot() {
+    kill -TERM $$
+    sleep 2m
+    echo "failed to reboot?" 1>&2
+    exit 1
+}
+
+# Determine our origin refspec - we'll use this as a test base
+rpmostree=$(which rpm-ostree 2>/dev/null)
+if test -z "${rpmostree}"; then
+    skip "no rpm-ostree, at some point point this to raw ostree too"
+fi
+
+# We need to be root
+assert_streq $(id -u) 0
+
+rpmostree_query_json() {
+    query=$1
+    rpm-ostree status --json | jq -r "${query}"
+}
+host_refspec=$(rpmostree_query_json '.deployments[0].origin')
+host_commit=$(rpmostree_query_json '.deployments[0].checksum')
+host_osname=$(rpmostree_query_json '.deployments[0].osname')
diff --git a/tests/kola/libtest-core.sh b/tests/kola/libtest-core.sh
new file mode 100644 (file)
index 0000000..945d285
--- /dev/null
@@ -0,0 +1,153 @@
+# Core source library for shell script tests; the
+# canonical version lives in:
+#
+#   https://github.com/ostreedev/ostree
+#
+# Known copies are in the following repos:
+#
+# - https://github.com/projectatomic/rpm-ostree
+#
+# Copyright (C) 2017 Colin Walters <walters@verbum.org>
+#
+# SPDX-License-Identifier: LGPL-2.0+
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the
+# Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+# Boston, MA 02111-1307, USA.
+
+fatal() {
+    echo $@ 1>&2; exit 1
+}
+# fatal() is shorter to type, but retain this alias
+assert_not_reached () {
+    fatal "$@"
+}
+
+# Some tests look for specific English strings. Use a UTF-8 version
+# of the C (POSIX) locale if we have one, or fall back to en_US.UTF-8
+# (https://sourceware.org/glibc/wiki/Proposals/C.UTF-8)
+#
+# If we can't find the locale command assume we have support for C.UTF-8
+# (e.g. musl based systems)
+if type -p locale >/dev/null; then
+    export LC_ALL=$(locale -a | grep -iEe '^(C|en_US)\.(UTF-8|utf8)$' | head -n1 || true)
+    if [ -z "${LC_ALL}" ]; then fatal "Can't find suitable UTF-8 locale"; fi
+else
+    export LC_ALL=C.UTF-8
+fi
+# A GNU extension, used whenever LC_ALL is not C
+unset LANGUAGE
+
+# This should really be the default IMO
+export G_DEBUG=fatal-warnings
+
+assert_streq () {
+    test "$1" = "$2" || fatal "$1 != $2"
+}
+
+assert_str_match () {
+    if ! echo "$1" | grep -E -q "$2"; then
+             fatal "$1 does not match regexp $2"
+    fi
+}
+
+assert_not_streq () {
+    (! test "$1" = "$2") || fatal "$1 == $2"
+}
+
+assert_has_file () {
+    test -f "$1" || fatal "Couldn't find '$1'"
+}
+
+assert_has_dir () {
+    test -d "$1" || fatal "Couldn't find '$1'"
+}
+
+# Dump ls -al + file contents to stderr, then fatal()
+_fatal_print_file() {
+    file="$1"
+    shift
+    ls -al "$file" >&2
+    sed -e 's/^/# /' < "$file" >&2
+    fatal "$@"
+}
+
+assert_not_has_file () {
+    if test -f "$1"; then
+        _fatal_print_file "$1" "File '$1' exists"
+    fi
+}
+
+assert_not_file_has_content () {
+    fpath=$1
+    shift
+    for re in "$@"; do
+        if grep -q -e "$re" "$fpath"; then
+            _fatal_print_file "$fpath" "File '$fpath' matches regexp '$re'"
+        fi
+    done
+}
+
+assert_not_has_dir () {
+    if test -d "$1"; then
+             fatal "Directory '$1' exists"
+    fi
+}
+
+assert_file_has_content () {
+    fpath=$1
+    shift
+    for re in "$@"; do
+        if ! grep -q -e "$re" "$fpath"; then
+            _fatal_print_file "$fpath" "File '$fpath' doesn't match regexp '$re'"
+        fi
+    done
+}
+
+assert_file_has_content_literal () {
+    fpath=$1; shift
+    for s in "$@"; do
+        if ! grep -q -F -e "$s" "$fpath"; then
+            _fatal_print_file "$fpath" "File '$fpath' doesn't match fixed string list '$s'"
+        fi
+    done
+}
+
+assert_file_has_mode () {
+    mode=$(stat -c '%a' $1)
+    if [ "$mode" != "$2" ]; then
+        fatal "File '$1' has wrong mode: expected $2, but got $mode"
+    fi
+}
+
+assert_symlink_has_content () {
+    if ! test -L "$1"; then
+        fatal "File '$1' is not a symbolic link"
+    fi
+    if ! readlink "$1" | grep -q -e "$2"; then
+        _fatal_print_file "$1" "Symbolic link '$1' doesn't match regexp '$2'"
+    fi
+}
+
+assert_file_empty() {
+    if test -s "$1"; then
+        _fatal_print_file "$1" "File '$1' is not empty"
+    fi
+}
+
+# Use to skip all of these tests
+skip() {
+    echo "1..0 # SKIP" "$@"
+    exit 0
+}
diff --git a/tests/kola/nondestructive/data b/tests/kola/nondestructive/data
new file mode 120000 (symlink)
index 0000000..a96aa0e
--- /dev/null
@@ -0,0 +1 @@
+..
\ No newline at end of file
diff --git a/tests/kola/nondestructive/itest-bare-unit.sh b/tests/kola/nondestructive/itest-bare-unit.sh
new file mode 100644 (file)
index 0000000..d399d78
--- /dev/null
@@ -0,0 +1,42 @@
+#!/bin/bash
+
+# Run test-basic.sh as root.
+# https://github.com/ostreedev/ostree/pull/1199
+
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+
+fatal "FIXME - need to also sync over the installed tests"
+
+date
+# These tests sort of bypass the installed-tests spec;
+# fixing that would require installing g-d-t-r, though
+# more ideally we architect things with a "control" container
+# distinct from the host.
+export G_TEST_SRCDIR=$(realpath $dn/../../..)
+
+# Use /var/tmp to hopefully use XFS + O_TMPFILE etc.
+prepare_tmpdir /var/tmp
+trap _tmpdir_cleanup EXIT
+/usr/libexec/installed-tests/libostree/test-basic.sh
+/usr/libexec/installed-tests/libostree/test-basic-c
+date
+
+# Test error message when opening a non-world-readable object
+# https://github.com/ostreedev/ostree/issues/1562
+rm repo files -rf
+chmod a+rx .
+ostree --repo=repo init --mode=bare
+mkdir files
+touch files/unreadable
+chmod 0 files/unreadable
+ostree --repo=repo commit -b testbranch --tree=dir=files
+# We should be able to read as root due to CAP_DAC_OVERRIDE
+ostree --repo=repo cat testbranch /unreadable >/dev/null
+if setpriv --reuid bin --regid bin --clear-groups ostree --repo=repo cat testbranch /unreadable 2>err.txt; then
+    fatal "Listed unreadable object as non-root"
+fi
+assert_file_has_content err.txt "Opening content object.*openat: Permission denied"
+
+date
diff --git a/tests/kola/nondestructive/itest-bare-user-root.sh b/tests/kola/nondestructive/itest-bare-user-root.sh
new file mode 100755 (executable)
index 0000000..ad3d2da
--- /dev/null
@@ -0,0 +1,40 @@
+#!/bin/bash
+
+# Tests of the "raw ostree" functionality using the host's ostree repo as uid 0.
+
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+
+echo "1..1"
+date
+
+prepare_tmpdir
+ostree --repo=repo init --mode=bare-user
+mkdir -p components/{dbus,systemd}/usr/{bin,lib}
+echo dbus binary > components/dbus/usr/bin/dbus-daemon
+chmod a+x components/dbus/usr/bin/dbus-daemon
+echo dbus lib > components/dbus/usr/lib/libdbus.so.1
+echo dbus helper > components/dbus/usr/lib/dbus-daemon-helper
+chmod a+x components/dbus/usr/lib/dbus-daemon-helper
+echo systemd binary > components/systemd/usr/bin/systemd
+chmod a+x components/systemd/usr/bin/systemd
+echo systemd lib > components/systemd/usr/lib/libsystemd.so.1
+
+# Make the gid on dbus 81 like fedora, also ensure no xattrs
+ostree --repo=repo commit --no-xattrs -b component-dbus --owner-uid 0 --owner-gid 81 --tree=dir=components/dbus
+ostree --repo=repo commit --no-xattrs -b component-systemd --owner-uid 0 --owner-gid 0 --tree=dir=components/systemd
+rm rootfs -rf
+for component in dbus systemd; do
+    ostree --repo=repo checkout -U -H component-${component} --union rootfs
+done
+echo 'some rootfs data' > rootfs/usr/lib/cache.txt
+# Commit using the host's selinux policy
+ostree --repo=repo commit --selinux-policy / -b rootfs --link-checkout-speedup --tree=dir=rootfs
+ostree --repo=repo ls rootfs /usr/bin/systemd >ls.txt
+assert_file_has_content ls.txt '^-007.. 0 0 .*/usr/bin/systemd'
+ostree --repo=repo ls -X rootfs /usr/lib/dbus-daemon-helper >ls.txt
+assert_file_has_content ls.txt '^-007.. 0 81 .*security\.selinux.*/usr/lib/dbus-daemon-helper'
+assert_not_file_has_content ls.txt 'user\.ostreemeta'
+echo "ok bare-user link-checkout-speedup with modified xattrs maintains uids"
+date
diff --git a/tests/kola/nondestructive/itest-bareuser-nouserxattrs.sh b/tests/kola/nondestructive/itest-bareuser-nouserxattrs.sh
new file mode 100755 (executable)
index 0000000..7908d31
--- /dev/null
@@ -0,0 +1,24 @@
+#!/bin/bash
+
+# Test that initializing a bare-user repo on tmpfs fails
+# Maybe at some point this will be fixed in the kernel
+# but I doubt it'll be soon
+# https://www.spinics.net/lists/linux-mm/msg109775.html
+
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+
+prepare_tmpdir
+trap _tmpdir_cleanup EXIT
+date
+
+mkdir mnt
+mount -t tmpfs tmpfs mnt
+if ostree --repo=mnt/repo init --mode=bare-user 2>err.txt; then
+    umount mnt
+    assert_not_reached "bare-user on tmpfs worked?"
+fi
+umount mnt
+assert_file_has_content err.txt "Operation not supported"
+date
diff --git a/tests/kola/nondestructive/itest-payload-link.sh b/tests/kola/nondestructive/itest-payload-link.sh
new file mode 100755 (executable)
index 0000000..6cfe291
--- /dev/null
@@ -0,0 +1,140 @@
+#!/bin/bash
+#
+# Copyright (C) 2018 Red Hat, Inc.
+#
+# SPDX-License-Identifier: LGPL-2.0+
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the
+# Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+# Boston, MA 02111-1307, USA.
+
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+
+echo "1..1"
+date
+
+# Use /var/tmp so we have O_TMPFILE etc.
+prepare_tmpdir /var/tmp
+trap _tmpdir_cleanup EXIT
+# We use this user down below, it needs access too
+setfacl -d -m u:bin:rwX .
+setfacl -m u:bin:rwX .
+ostree --repo=repo init --mode=archive
+echo -e '[archive]\nzlib-level=1\n' >> repo/config
+
+mkdir content
+cd content
+dd if=/dev/urandom of=bigobject bs=4k count=2560
+cp --reflink=auto bigobject bigobject2
+# Different metadata, same content
+chown bin:bin bigobject2
+cd ..
+ostree --repo=repo commit -b dupobjects --consume --selinux-policy=/ --tree=dir=content
+ostree --repo=repo summary -u
+
+run_tmp_webserver $(pwd)/repo
+
+origin=$(cat ${test_tmpdir}/httpd-address)
+
+cleanup() {
+    cd ${test_tmpdir}
+    for mnt in ${mnts:-}; do
+        umount ${mnt} || true
+    done
+    for blkdev in ${blkdevs:-}; do
+        losetup -d ${blkdev} || true
+    done
+}
+trap cleanup EXIT
+
+truncate -s 2G testblk1.img
+blkdev1=$(losetup --find --show $(pwd)/testblk1.img)
+blkdevs="${blkdev1}"
+# This filesystem must support reflinks
+mkfs.xfs -m reflink=1 ${blkdev1}
+mkdir mnt1
+mount ${blkdev1} mnt1
+mnts=mnt1
+
+truncate -s 2G testblk2.img
+blkdev2=$(losetup --find --show $(pwd)/testblk2.img)
+blkdevs="${blkdev1} ${blkdev2}"
+mkfs.xfs -m reflink=1 ${blkdev2}
+mkdir mnt2
+mount ${blkdev2} mnt2
+mnts="mnt1 mnt2"
+
+cd mnt1
+# See above for setfacl rationale
+setfacl -d -m u:bin:rwX .
+setfacl -m u:bin:rwX .
+ls -al .
+runuser -u bin mkdir foo
+runuser -u bin touch foo/bar
+ls -al foo
+
+# Test that reflink is really there (not just --reflink=auto)
+touch a
+cp --reflink a b
+mkdir repo
+ostree --repo=repo init
+ostree config --repo=repo set core.payload-link-threshold 0
+ostree --repo=repo remote add origin --set=gpg-verify=false ${origin}
+ostree --repo=repo pull --disable-static-deltas origin dupobjects
+find repo -type l -name '*.payload-link' >payload-links.txt
+assert_streq "$(wc -l < payload-links.txt)" "1"
+
+cat payload-links.txt | while read i; do
+    payload_checksum=$(basename $(dirname $i))$(basename $i .payload-link)
+    payload_checksum_calculated=$(sha256sum $(readlink -f $i) | cut -d ' ' -f 1)
+    assert_streq "${payload_checksum}" "${payload_checksum_calculated}"
+done
+echo "ok payload link"
+
+ostree --repo=repo checkout dupobjects content
+# And another object which differs just in metadata
+cp --reflink=auto content/bigobject{,3}
+chown operator:0 content/bigobject3
+cat >unpriv-child-repo.sh <<EOF
+#!/bin/bash
+# Check that commit to an user repo that has a parent still works
+set -xeuo pipefail
+ostree --repo=child-repo init --mode=bare-user
+ostree --repo=child-repo remote add origin --set=gpg-verify=false ${origin}
+ostree --repo=child-repo config set core.parent $(pwd)/repo
+ostree --repo=child-repo commit -b test content
+EOF
+chmod a+x unpriv-child-repo.sh
+runuser -u bin ./unpriv-child-repo.sh
+find child-repo -type l -name '*.payload-link' >payload-links.txt
+assert_streq "$(wc -l < payload-links.txt)" "0"
+rm content -rf
+
+echo "ok reflink unprivileged with parent repo"
+
+# We can't reflink across devices though
+cd ../mnt2
+ostree --repo=repo init --mode=archive
+ostree --repo=repo config set core.parent $(cd ../mnt1/repo && pwd)
+ostree --repo=../mnt1/repo checkout dupobjects content
+ostree --repo=repo commit -b dupobjects2 --consume --tree=dir=content
+ostree --repo=repo pull --disable-static-deltas origin dupobjects
+find repo -type l -name '*.payload-link' >payload-links.txt
+assert_streq "$(wc -l < payload-links.txt)" "0"
+
+echo "ok payload link across devices"
+
+date
diff --git a/tests/kola/nondestructive/itest-pull-space.sh b/tests/kola/nondestructive/itest-pull-space.sh
new file mode 100755 (executable)
index 0000000..97524f6
--- /dev/null
@@ -0,0 +1,116 @@
+#!/bin/bash
+# Test min-free-space-percent using loopback devices
+
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+date
+
+prepare_tmpdir
+truncate -s 20MB testblk.img
+blkdev=$(losetup --find --show $(pwd)/testblk.img)
+mkfs.xfs ${blkdev}
+mkdir mnt
+mount ${blkdev} mnt
+
+# first test min-free-space-percent
+ostree --repo=mnt/repo init --mode=bare-user
+echo 'fsync=false' >> mnt/repo/config
+if ostree --repo=mnt/repo pull-local /ostree/repo ${host_commit} 2>err.txt; then
+    fatal "succeeded in doing a pull with no free space"
+fi
+assert_file_has_content err.txt "min-free-space-percent"
+echo "ok min-free-space-percent"
+
+# now test min-free-space-size
+rm -rf mnt/repo
+ostree --repo=mnt/repo init --mode=bare-user
+echo 'fsync=false' >> mnt/repo/config
+echo 'min-free-space-size=10MB' >> mnt/repo/config
+if ostree --repo=mnt/repo pull-local /ostree/repo ${host_commit} 2>err.txt; then
+    fatal "succeeded in doing a pull with no free space"
+fi
+assert_file_has_content err.txt "min-free-space-size"
+echo "ok min-free-space-size (error)"
+
+
+# min-free-space-size success
+ostree --repo=repo init --mode=bare-user
+echo 'fsync=false' >> repo/config
+echo 'min-free-space-size=1MB' >> repo/config
+ostree --repo=repo pull-local /ostree/repo ${host_commit}
+echo "ok min-free-space-size (success)"
+
+# metadata object write should succeed even if free space is lower than
+# min-free-space value
+rm -rf mnt/repo
+ostree --repo=mnt/repo init --mode=bare-user
+echo 'fsync=false' >> mnt/repo/config
+echo 'min-free-space-size=10MB' >> mnt/repo/config
+ostree --repo=mnt/repo pull-local --commit-metadata-only /ostree/repo ${host_commit}
+echo "ok metadata write even when free space is lower than min-free-space value"
+
+rm -rf mnt/repo
+
+# Test min-free-space-size on deltas
+
+#helper function copied from test-delta.sh
+get_assert_one_direntry_matching() {
+    local path=$1
+    local r=$2
+    local child=""
+    local bn
+    for p in ${path}/*; do
+  bn=$(basename $p)
+  if ! echo ${bn} | grep -q "$r"; then
+      continue
+  fi
+  if test -z "${child}"; then
+      child=${bn}
+  else
+      assert_not_reached "Expected only one child matching ${r} in ${path}";
+  fi
+    done
+    if test -z "${child}"; then
+  assert_not_reached "Failed to find child matching ${r}"
+    fi
+    echo ${child}
+}
+
+mkdir mnt/repo1 mnt/repo2
+ostree --repo=mnt/repo1 init --mode=bare-user
+ostree --repo=mnt/repo2 init --mode=bare-user
+mkdir files
+echo "hello" >> files/test.txt
+truncate -s 2MB files/test.txt
+
+host_commit=$(ostree --repo=mnt/repo1 commit -b test -s test --tree=dir=files)
+
+origrev=$(ostree --repo=mnt/repo1 rev-parse test)
+ostree --repo=mnt/repo1 static-delta generate --empty --to=${origrev}
+echo 'fsync=false' >> mnt/repo2/config
+echo 'min-free-space-size=12MB' >> mnt/repo2/config
+
+deltaprefix=$(get_assert_one_direntry_matching mnt/repo1/deltas '.')
+deltadir=$(get_assert_one_direntry_matching mnt/repo1/deltas/${deltaprefix} '.')
+
+# Try to pull delta and trigger error
+if ostree --repo=mnt/repo2 static-delta apply-offline mnt/repo1/deltas/${deltaprefix}/${deltadir} 2>err.txt; then
+    fatal "succeeded in doing a delta pull with no free space"
+fi
+assert_file_has_content err.txt "min-free-space-size"
+echo "OK min-free-space-size delta pull (error)"
+
+# Re-adjust min-free-space-size so that delta pull succeeds
+sed -i s/min-free-space-size=12MB/min-free-space-size=1MB/g mnt/repo2/config
+rm -rf mnt/repo2/deltas
+ostree --repo=mnt/repo2 static-delta apply-offline mnt/repo1/deltas/${deltaprefix}/${deltadir}
+
+echo "OK min-free-space-size delta pull (success)"
+rm -rf files
+
+umount mnt
+losetup -d ${blkdev}
+rm testblk.img
+
+date
diff --git a/tests/kola/nondestructive/itest-pull.sh b/tests/kola/nondestructive/itest-pull.sh
new file mode 100755 (executable)
index 0000000..770f244
--- /dev/null
@@ -0,0 +1,82 @@
+#!/bin/bash
+
+# Using the host ostree, test HTTP pulls
+
+set -xeuo pipefail
+
+# FIXME: https://github.com/ostreedev/ostree/pull/1548
+exit 0
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+date
+
+prepare_tmpdir /var/tmp
+trap _tmpdir_cleanup EXIT
+
+# Take the host's ostree, and make it archive
+mkdir repo
+ostree --repo=repo init --mode=archive
+echo -e '[archive]\nzlib-level=1\n' >> repo/config
+host_nonremoteref=$(echo ${host_refspec} | sed 's,[^:]*:,,')
+log_timestamps() {
+    date
+    "$@"
+    date
+}
+log_timestamps ostree --repo=repo pull-local /ostree/repo ${host_commit}
+ostree --repo=repo refs ${host_commit} --create=${host_nonremoteref}
+
+run_tmp_webserver $(pwd)/repo
+# Now test pulling via HTTP (no deltas) to a new bare-user repo
+ostree --repo=bare-repo init --mode=bare-user
+ostree --repo=bare-repo remote add origin --set=gpg-verify=false $(cat ${test_tmpdir}/httpd-address)
+log_timestamps ostree --repo=bare-repo pull --disable-static-deltas origin ${host_nonremoteref}
+
+echo "ok pull"
+
+# fsck marks commits partial
+# https://github.com/ostreedev/ostree/pull/1533
+for d in $(find bare-repo/objects/ -maxdepth 1 -type d); do
+    (find ${d} -name '*.file' || true) | head -20 | xargs rm -f
+done
+date
+if ostree --repo=bare-repo fsck |& tee fsck.txt; then
+    fatal "fsck unexpectedly succeeded"
+fi
+date
+assert_streq $(grep -cE -e 'Marking commit as partial' fsck.txt) "1"
+log_timestamps ostree --repo=bare-repo pull origin ${host_nonremoteref}
+# Don't need a full fsck here
+ostree --repo=bare-repo ls origin:${host_nonremoteref} >/dev/null
+
+rm bare-repo repo -rf
+
+# Try copying the host's repo across a mountpoint for direct
+# imports.
+cd ${test_tmpdir}
+mkdir tmpfs mnt
+mount --bind tmpfs mnt
+cd mnt
+ostree --repo=repo init --mode=bare
+log_timestamps ostree --repo=repo pull-local /ostree/repo ${host_commit}
+log_timestamps ostree --repo=repo fsck
+cd ..
+
+# Also, we shouldn't copy xattrs on metadata objects
+commit_path=objects/${host_commit:0:2}/${host_commit:2}.commit
+ostree --repo=testarchive init --mode=archive
+ostree --repo=testarchive pull-local --commit-metadata-only /ostree/repo ${host_commit}
+setfattr -n user.ostreetesting -v hello testarchive/${commit_path}
+ostree --repo=mnt/testarchive2 init --mode=archive
+ostree --repo=mnt/testarchive2 pull-local --commit-metadata-only testarchive ${host_commit}
+if getfattr -m user.ostreetesting mnt/testarchive2/${commit_path} 2>/dev/null; then
+    fatal "copied metadata xattr"
+fi
+echo "ok no metadata xattr copy"
+
+umount mnt
+
+# Cleanup
+kill -TERM $(cat ${test_tmpdir}/httpd-pid)
+echo "ok"
+date
diff --git a/tests/kola/nondestructive/itest-remotes.sh b/tests/kola/nondestructive/itest-remotes.sh
new file mode 100755 (executable)
index 0000000..d1fb455
--- /dev/null
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+# Test that we didn't regress /etc/ostree/remotes.d handling
+
+set -xeuo pipefail
+
+. ${KOLA_EXT_DATA}/libinsttest.sh
+date
+
+prepare_tmpdir
+trap _tmpdir_cleanup EXIT
+
+ostree remote list > remotes.txt
+if ! test -s remotes.txt; then
+    assert_not_reached "no ostree remotes"
+fi
+date
diff --git a/tests/kola/nondestructive/libtest-core.sh b/tests/kola/nondestructive/libtest-core.sh
new file mode 120000 (symlink)
index 0000000..d26203e
--- /dev/null
@@ -0,0 +1 @@
+../libtest-core.sh
\ No newline at end of file
deleted file mode 100644 (file)
index 945d2857240eb1ef21125c0294cee5d42ef5090c..0000000000000000000000000000000000000000
+++ /dev/null
@@ -1,153 +0,0 @@
-# Core source library for shell script tests; the
-# canonical version lives in:
-#
-#   https://github.com/ostreedev/ostree
-#
-# Known copies are in the following repos:
-#
-# - https://github.com/projectatomic/rpm-ostree
-#
-# Copyright (C) 2017 Colin Walters <walters@verbum.org>
-#
-# SPDX-License-Identifier: LGPL-2.0+
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library; if not, write to the
-# Free Software Foundation, Inc., 59 Temple Place - Suite 330,
-# Boston, MA 02111-1307, USA.
-
-fatal() {
-    echo $@ 1>&2; exit 1
-}
-# fatal() is shorter to type, but retain this alias
-assert_not_reached () {
-    fatal "$@"
-}
-
-# Some tests look for specific English strings. Use a UTF-8 version
-# of the C (POSIX) locale if we have one, or fall back to en_US.UTF-8
-# (https://sourceware.org/glibc/wiki/Proposals/C.UTF-8)
-#
-# If we can't find the locale command assume we have support for C.UTF-8
-# (e.g. musl based systems)
-if type -p locale >/dev/null; then
-    export LC_ALL=$(locale -a | grep -iEe '^(C|en_US)\.(UTF-8|utf8)$' | head -n1 || true)
-    if [ -z "${LC_ALL}" ]; then fatal "Can't find suitable UTF-8 locale"; fi
-else
-    export LC_ALL=C.UTF-8
-fi
-# A GNU extension, used whenever LC_ALL is not C
-unset LANGUAGE
-
-# This should really be the default IMO
-export G_DEBUG=fatal-warnings
-
-assert_streq () {
-    test "$1" = "$2" || fatal "$1 != $2"
-}
-
-assert_str_match () {
-    if ! echo "$1" | grep -E -q "$2"; then
-             fatal "$1 does not match regexp $2"
-    fi
-}
-
-assert_not_streq () {
-    (! test "$1" = "$2") || fatal "$1 == $2"
-}
-
-assert_has_file () {
-    test -f "$1" || fatal "Couldn't find '$1'"
-}
-
-assert_has_dir () {
-    test -d "$1" || fatal "Couldn't find '$1'"
-}
-
-# Dump ls -al + file contents to stderr, then fatal()
-_fatal_print_file() {
-    file="$1"
-    shift
-    ls -al "$file" >&2
-    sed -e 's/^/# /' < "$file" >&2
-    fatal "$@"
-}
-
-assert_not_has_file () {
-    if test -f "$1"; then
-        _fatal_print_file "$1" "File '$1' exists"
-    fi
-}
-
-assert_not_file_has_content () {
-    fpath=$1
-    shift
-    for re in "$@"; do
-        if grep -q -e "$re" "$fpath"; then
-            _fatal_print_file "$fpath" "File '$fpath' matches regexp '$re'"
-        fi
-    done
-}
-
-assert_not_has_dir () {
-    if test -d "$1"; then
-             fatal "Directory '$1' exists"
-    fi
-}
-
-assert_file_has_content () {
-    fpath=$1
-    shift
-    for re in "$@"; do
-        if ! grep -q -e "$re" "$fpath"; then
-            _fatal_print_file "$fpath" "File '$fpath' doesn't match regexp '$re'"
-        fi
-    done
-}
-
-assert_file_has_content_literal () {
-    fpath=$1; shift
-    for s in "$@"; do
-        if ! grep -q -F -e "$s" "$fpath"; then
-            _fatal_print_file "$fpath" "File '$fpath' doesn't match fixed string list '$s'"
-        fi
-    done
-}
-
-assert_file_has_mode () {
-    mode=$(stat -c '%a' $1)
-    if [ "$mode" != "$2" ]; then
-        fatal "File '$1' has wrong mode: expected $2, but got $mode"
-    fi
-}
-
-assert_symlink_has_content () {
-    if ! test -L "$1"; then
-        fatal "File '$1' is not a symbolic link"
-    fi
-    if ! readlink "$1" | grep -q -e "$2"; then
-        _fatal_print_file "$1" "Symbolic link '$1' doesn't match regexp '$2'"
-    fi
-}
-
-assert_file_empty() {
-    if test -s "$1"; then
-        _fatal_print_file "$1" "File '$1' is not empty"
-    fi
-}
-
-# Use to skip all of these tests
-skip() {
-    echo "1..0 # SKIP" "$@"
-    exit 0
-}
new file mode 120000 (symlink)
index 0000000000000000000000000000000000000000..11131fcadc8a0bc788d10f53edaeca9e27f1180e
--- /dev/null
@@ -0,0 +1 @@
+kola/libtest-core.sh
\ No newline at end of file